Auto merge of #83152 - guswynn:jemallocator_part2, r=Mark-Simulacrum
Use tikv-jemallocator in rustc/rustdoc in addition to jemalloc-sys when enabled.
In https://github.com/rust-lang/rust/pull/81782 it was mentioned that one reason rustc may benefit from minimalloc is it doesn't use the `sdallocx` api from jemalloc.
Currently, on unix, rust uses jemalloc by importing its symbols to use them with the default, System (libc) global allocator.
This PR switches its global alloc to `tikv-jemallocator`, which correctly uses sized deallocation (https://docs.rs/tikv-jemallocator/0.4.1/src/tikv_jemallocator/lib.rs.html#121-126). `tikv-jemallocator`, as far as I can tell, is a more up-to-date set of bindings to jemalloc than `jemallocator`
The perf results of this pr are in large part due to the version upgrade of jemalloc, but sized deallocation has a non-trivial improvement, particularly to rustdoc.
This pr also includes changes to bootstrap to correctly pass the jemalloc feature through to the rustdoc build
On recursive types such as in the example given in https://github.com/rust-lang/rust/issues/83150, the current implementation of `LateBoundRegionNameCollector` has very long compilation runtimes. To prevent those we store the types visited in the `middle::ty::Visitor` implementation of `LateBoundRegionNameCollector` in a `SsoHashSet`.
Auto merge of #83890 - Dylan-DPC:rollup-9fqy3fe, r=Dylan-DPC
Rollup of 5 pull requests
Successful merges:
- #83368 (Add `download-rustc = "if-unchanged"`)
- #83705 (Give a better error when --theme is not a CSS file)
- #83853 (Disallow the use of high byte registes as operands on x86_64)
- #83877 (Remove unnecessary exceptions to the platform-specific code check)
- #83878 (Fix racing file access in tidy)
Rollup merge of #83878 - the8472:fix-concurrent-tidy-access, r=Mark-Simulacrum
Fix racing file access in tidy
That should fix the failure in https://github.com/rust-lang/rust/pull/83776#issuecomment-813311289
The file is only created for a brief moment during the bins checks in the source directories while other checks may also be visiting the same directory. By skipping it we avoid file not found errors.
Rollup merge of #83877 - jyn514:exceptions, r=Mark-Simulacrum
Remove unnecessary exceptions to the platform-specific code check
Some of these were just wrong, like src/librustc. Some looked outdated,
like std::f64. Not sure what was going on with the others - maybe this
check isn't as smart as it needs to be? But in the meantime it seems
silly to ignore the check if it will pass anyway.
Auto merge of #83880 - Dylan-DPC:rollup-hz9ph0e, r=Dylan-DPC
Rollup of 8 pull requests
Successful merges:
- #81922 (Let `#[allow(unstable_name_collisions)]` work for things other than function)
- #82483 (Use FromStr trait for number option parsing)
- #82739 (Use the beta compiler for building bootstrap tools when `download-rustc` is set)
- #83650 (Update Source Serif to release 4.004)
- #83826 (List trait impls before deref methods in doc's sidebar)
- #83831 (Add `#[inline]` to IpAddr methods)
- #83863 (Render destructured struct function param names as underscore)
- #83865 (Don't report disambiguator error if link would have been ignored)
Rollup merge of #83865 - camelid:disamb-err-fix, r=jyn514
Don't report disambiguator error if link would have been ignored
Fixes #83859.
This prevents us from warning on links such as `<hello@example.com>`.
Note that we still warn on links such as `<hello@localhost>` because
they have no dots in them. However, the links will still work, even
though a warning is reported.
Rollup merge of #83826 - slightlyoutofphase:rustdoc-sidebar-order-shuffle, r=jyn514
List trait impls before deref methods in doc's sidebar
This PR is acting directly on a suggestion made by ```````@jyn514``````` in #83133. I've tested the changes locally, and can confirm that it does in fact properly achieve what he thought it would. This PR also in turn closes #83133.
Rollup merge of #82739 - jyn514:separate-stage0-stage1, r=Mark-Simulacrum
Use the beta compiler for building bootstrap tools when `download-rustc` is set
## Motivation
This avoids having to rebuild bootstrap and tidy each time you rebase
over master. In particular, it makes rebasing and running `x.py fmt` on
each commit in a branch significantly faster. It also avoids having to
rebuild bootstrap after setting `download-rustc = true`.
## Implementation
Instead of extracting the CI artifacts directly to `stage0/`, extract
them to `ci-rustc/` instead. Continue to copy them to the proper
sysroots as necessary for all stages except stage 0.
This also requires `bootstrap.py` to download both stage0 and CI
artifacts and distinguish between the two when checking stamp files.
Note that since tools have to be built by the same compiler that built
`rustc-dev` and the standard library, the downloaded artifacts can't be
reused when building with the beta compiler. To make sure this is still
a good user experience, warn when building with the beta compiler, and
default to building with stage 2.
I tested this by rebasing this PR from edeee915b1c52f97411e57ef6b1a8bd46548a37a over 1c77a1fa3ca574f2a40056f64d498db8efe0d8a8 and confirming that only the bootstrap library itself had to be rebuilt, not any dependencies and not `tidy`. I also tested that a clean build with `x.py build` builds rustdoc exactly once and does no other work, and that `touch src/librustdoc/lib.rs && x.py build` works. `x.py check` still behaves as before (checks using the beta compiler, even if there are changes to `compiler/`).
Helps with https://github.com/rust-lang/rust/issues/81930.
Rollup merge of #81922 - magurotuna:issue81522, r=matthewjasper
Let `#[allow(unstable_name_collisions)]` work for things other than function
Fixes #81522
In addition to the report in #81522, currently `#[allow(unstable_name_collisions)]` doesn't suppress the corresponding diagnostics even if this attribute is appended to an expression statement or a let statement. It seems like this is because the wrong `HirId` is passed to `struct_span_lint_hir`.
It's fixed in this PR, and a regression test for it is also added.
this file is only created for a brief moment during the bins checks
in the source directories while other checks may also be visiting
that directory. skip processing it to avoid missing file errors
Joshua Nelson [Mon, 5 Apr 2021 10:02:29 +0000 (06:02 -0400)]
Remove unnecessary exceptions to the platform-specific code check
Some of these were just wrong, like src/librustc. Some looked outdated,
like std::f64. Not sure what was going on with the others - maybe this
check isn't as smart as it needs to be? But it the meantime it seems
silly to ignore the check if it will pass anyway.
Auto merge of #83858 - joshtriplett:unsafe-cell-always-inline, r=Mark-Simulacrum
Use `#[inline(always)]` on trivial UnsafeCell methods
UnsafeCell is the standard building block for shared mutable data
structures. UnsafeCell should add zero overhead compared to using raw
pointers directly.
Some reports suggest that debug builds, or even builds at opt-level 1,
may not always be inlining its methods. Mark the methods as
`#[inline(always)]`, since once inlined the methods should result in no
actual code other than field accesses.
Don't report disambiguator error if link would have been ignored
This prevents us from warning on links such as `<hello@example.com>`.
Note that we still warn on links such as `<hello@localhost>` because
they have no dots in them. However, the links will still work, even
though a warning is reported.
The attribute could always be replaced with `-C link-arg`, but cargo didn't provide a reasonable way to pass such flags to rustc.
Now cargo supports `cargo:rustc-link-arg*` directives in build scripts (https://doc.rust-lang.org/cargo/reference/unstable.html#extra-link-arg), so this attribute can be removed.
In the codegen and codegen-units test suites, the `//` comment markers
were kept in order not to affect any source locations. This is because
these tests cannot be automatically `--bless`ed.
Rollup merge of #83019 - eddyb:spirv-no-block-swap, r=nagisa
core: disable `ptr::swap_nonoverlapping_one`'s block optimization on SPIR-V.
SPIR-V primarily supports what it calls the "Logical addressing model" (and AFAIK for graphical shaders it's the only option), and what that implies is that there is no "memory" to uniformly address at some byte/word level, and that you can't really talk about values having a "raw representation" in terms of sequences of bytes. Therefore, the "block"-wise swapping optimization employed by `ptr::swap_nonoverlapping_one` (where a "block" is 32 bytes, currently), is fundamentally incompatible with SPIR-V "memory".
As such, [Rust-GPU](https://github.com/EmbarkStudios/rust-gpu/)'s `rustc_codegen_spirv` backend cannot currently allow the use of `ptr::swap_nonoverlapping_one` - but that comes at a great price, since it's the building block of `mem::{swap,replace}`, and those in turn are used by e.g. `Option::take` and `Range`'s `Iterator` implementation (the latter blocking the use of `for i in 0..n` loops).
There's 4 options I can see in terms of supporting `ptr::swap_nonoverlapping_one` in `rustc_codegen_spirv`:
* legalize the block-wise swap loop back into swapping whole values, for SPIR-V
* this is made borderline impossible by the fact that the size of the state "on the stack" is a block, and has to be expanded back to the appropriate size of the value being swapped, so in practice this would have to effectively pattern-match on the exact shape of the block-wise swapping algorithm, as a roundabout way of "patching `core::ptr` on the fly"
* (**this PR**) disable the block-wise swap optimization altogether when `#[cfg(target_arch = "spirv")`
* I've tested it and it does in fact allow compiling `for i in 0..n` loops, which was my primary motivation
* main downside IMO is the fact that `core` now acknowledges an out-of-tree backend
* as a counterpoint, any attempt to compile Rust to SPIR-V would run into this problem, one way or another
* only enable the block-wise swap optimization on targets where it's been empirically proven to be an improvement
* would avoid any surprises in terms of potentially-broken/inefficient codegen, in general
* however, it may be universally applicable (thanks to caches), even if the optimal block size could differ
* move low-level swapping into an intrinsic, where the backend can choose any optimization approach it wants
* this also has an impact on MIR optimizations (cc ``@rust-lang/wg-mir-opt)`` - which currently cannot hope to make sense of e.g. `Option::take` despite it being effectively `_0 = *_1;` `*_1 = None;` `return;`
* long-term this is my preferred approach, and I can start working on it if that's desired, but I wanted to confirm that this swapping optimization is the final blocker for [Rust-GPU](https://github.com/EmbarkStudios/rust-gpu/) supporting e.g. range `for` loops
Auto merge of #82907 - petrochenkov:dercache, r=Aaron1011
resolve/expand: Cache intermediate results of `#[derive]` expansion
Expansion function for `#[derive]` (`rustc_builtin_macros::derive::Expander::expand`) may return an indeterminate result, and therefore can be called multiple times.
Previously we parsed the `#[derive(Foo, Bar)]`'s input and tried to resolve `Foo` and `Bar` on every such call.
Now we maintain a cache `Resolver::derive_data` and take all the necessary data from it if it was computed previously.
So `Foo, Bar` is now parsed at most once, and `Foo` and `Bar` are successfully resolved at most once.
Joshua Nelson [Wed, 10 Feb 2021 05:33:17 +0000 (00:33 -0500)]
Use the beta compiler for building bootstrap tools when `download-rustc` is set
## Motivation
This avoids having to rebuild bootstrap and tidy each time you rebase
over master. In particular, it makes rebasing and running `x.py fmt` on
each commit in a branch significantly faster. It also avoids having to
rebuild bootstrap after setting `download-rustc = true`.
## Implementation
Instead of extracting the CI artifacts directly to `stage0/`, extract
them to `ci-rustc/` instead. Continue to copy them to the proper
sysroots as necessary for all stages except stage 0.
This also requires `bootstrap.py` to download both stage0 and CI
artifacts and distinguish between the two when checking stamp files.
Note that since tools have to be built by the same compiler that built
`rustc-dev` and the standard library, the downloaded artifacts can't be
reused when building with the beta compiler. To make sure this is still
a good user experience, warn when building with the beta compiler, and
default to building with stage 2.
Mark Rousskov [Sun, 4 Apr 2021 18:51:22 +0000 (14:51 -0400)]
Workaround increased cache clearing in Cargo
1.52 Cargo adds rust-lang/cargo#8640 which means that cargo will try to purge
the doc directory caches for us. In theory this may mean that we can jettison
the clear_if_dirty for rustdoc versioning entirely, but for now just workaround
the effects of this change in a less principled but more local way.
Use `#[inline(always)]` on trivial UnsafeCell methods
UnsafeCell is the standard building block for shared mutable data
structures. UnsafeCell should add zero overhead compared to using raw
pointers directly.
Some reports suggest that debug builds, or even builds at opt-level 1,
may not always be inlining its methods. Mark the methods as
`#[inline(always)]`, since once inlined the methods should result in no
actual code other than field accesses.
Auto merge of #83855 - Dylan-DPC:rollup-oww62sh, r=Dylan-DPC
Rollup of 8 pull requests
Successful merges:
- #73945 (Add an unstable --json=unused-externs flag to print unused externs)
- #81619 (Implement `SourceIterator` and `InPlaceIterable` for `ResultShunt`)
- #82726 (BTree: move blocks around in node.rs)
- #83521 (2229: Fix diagnostic issue when using FakeReads in closures)
- #83532 (Fix compiletest on FreeBSD)
- #83793 (rustdoc: highlight macros more efficiently)
- #83809 (Remove unneeded INITIAL_IDS const)
- #83827 (cleanup leak after test to make miri happy)
Rollup merge of #83809 - GuillaumeGomez:remove-initial-ids, r=camelid
Remove unneeded INITIAL_IDS const
Some IDs inside this map didn't exist anymore, some others were duplicates of what we have inside `IdMap`. So instead of keeping the two around and since `INITIAL_IDS` was only used by `IdMap`, no need to keep both of them.
Rollup merge of #83521 - sexxi-goose:quick-diagnostic-fix, r=nikomatsakis
2229: Fix diagnostic issue when using FakeReads in closures
This PR fixes a diagnostic issue caused by https://github.com/rust-lang/rust/pull/82536. A temporary work around was used in this merged PR which involved feature gating the addition of FakeReads introduced as a result of pattern matching in closures.
The fix involves adding an optional closure DefId to ForLet and ForMatchedPlace FakeReadCauses. This DefId will only be added if a closure pattern matches a Place starting with an Upvar.
Rollup merge of #82726 - ssomers:btree_node_rearange, r=Mark-Simulacrum
BTree: move blocks around in node.rs
Without changing any names or implementation, reorder some members:
- Move down the ones defined long ago on the demised `struct Root`, to below the definition of their current host `struct NodeRef`.
- Move up some defined on `struct NodeRef` that are interspersed with those defined on `struct Handle`.
- Move up the `correct_…` methods squeezed between the two flavours of `push`.
- Move the unchecked static downcasts (`cast_to_…`) after the upcasts (`forget_`) and the (weirdly named) dynamic downcasts (`force`).
r? ````@Mark-Simulacrum````
Rollup merge of #73945 - est31:unused_externs, r=Mark-Simulacrum
Add an unstable --json=unused-externs flag to print unused externs
This adds an unstable flag to print a list of the extern names not used by cargo.
This PR will enable cargo to collect unused dependencies from all units and provide warnings.
The companion PR to cargo is: https://github.com/rust-lang/cargo/pull/8437
The goal is eventual stabilization of this flag in rustc as well as in cargo.
Discussion of this feature is mostly contained inside these threads: #57274 #72342 #72603
The feature builds upon the internal datastructures added by #72342
Externs are uniquely identified by name and the information is sufficient for cargo.
If the mode is enabled, rustc will print json messages like:
For a crate that got passed byteorder, openssl and webpki dependencies but needed none of them.
### Q: Why not pass -Wunused-crate-dependencies?
A: See [ehuss's comment here](https://github.com/rust-lang/rust/issues/57274#issuecomment-624839355)
TLDR: it's cleaner. Rust's warning system wasn't built to be filtered or edited by cargo.
Even a basic implementation of the feature would have to change the "n warnings emitted" line that rustc prints at the end.
Cargo ideally wants to synthesize its own warnings anyways. For example, it would be hard for rustc to emit warnings like
"dependency foo is only used by dev targets", suggesting to make it a dev-dependency instead.
### Q: Make rustc emit used or unused externs?
A: Emitting used externs has the advantage that it simplifies cargo's collection job.
However, emitting unused externs creates less data to be communicated between rustc and cargo.
Often you want to paste a cargo command obtained from `cargo build -vv` for doing something
completely unrelated. The message is emitted always, even if no warning or error is emitted.
At that point, even this tiny difference in "noise" matters. That's why I went with emitting unused externs.
### Q: One json msg per extern or a collective json msg?
A: Same as above, the data format should be concise. Having 30 lines for the 30 crates a crate uses would be disturbing to readers.
Also it helps the cargo implementation to know that there aren't more unused deps coming.
### Q: Why use names of externs instead of e.g. paths?
A: Names are both sufficient as well as neccessary to uniquely identify a passed `--extern` arg.
Names are sufficient because you *must* pass a name when passing an `--extern` arg.
Passing a path is optional on the other hand so rustc might also figure out a crate's location from the file system.
You can also put multiple paths for the same extern name, via e.g. `--extern hello=/usr/lib/hello.rmeta --extern hello=/usr/local/lib/hello.rmeta`,
but rustc will only ever use one of those paths.
Also, paths don't identify a dependency uniquely as it is possible to have multiple different extern names point to the same path.
So paths are ill-suited for identification.
### Q: What about 2015 edition crates?
A: They are fully supported.
Even on the 2015 edition, an explicit `--extern` flag is is required to enable `extern crate foo;` to work (outside of sysroot crates, which this flag doesn't warn about anyways).
So the lint would still fire on 2015 edition crates if you haven't included a dependency specified in Cargo.toml using `extern crate foo;` or similar.
The lint won't fire if your sole use in the crate is through a `extern crate foo;` statement, but that's not its job.
For detecting unused `extern crate foo` statements, there is the `unused_extern_crates` lint
which can be enabled by `#![warn(unused_extern_crates)]` or similar.
cc ```@jsgf``` ```@ehuss``` ```@petrochenkov``` ```@estebank```
Auto merge of #83451 - GuillaumeGomez:fix-error-code-tidy-check, r=Mark-Simulacrum
Fix error codes check run and ensure it will not go unnoticed again
Fixes #83268.
The error codes explanations were not checked anymore. I fixed this issue and also added variables to ensure that this won't happen again (at least not silently).
Auto merge of #82347 - the8472:parallelize-tidy, r=Mark-Simulacrum
Parallelize tidy
Split off from #81833
While that PR brings wall time of `x.py test tidy` down to 0m2.847s adding this one on top should bring it down to 0m1.673s.
r? `@Mark-Simulacrum`
Previous concerns can be found at https://github.com/rust-lang/rust/pull/81833#issuecomment-782754685 and https://github.com/rust-lang/rust/pull/81833#discussion_r575194633
Auto merge of #83267 - ssomers:btree_prune_range_search_overlap, r=Mark-Simulacrum
BTree: no longer search arrays twice to check Ord
A possible addition to / partial replacement of #83147: no longer linearly search the upper bound of a range in the initial portion of the keys we already know are below the lower bound.
- Should be faster: fewer key comparisons at the cost of some instructions dealing with offsets
- Makes code a little more complicated.
- No longer detects ill-defined `Ord` implementations, but that wasn't a publicised feature, and was quite incomplete, and was only done in the `range` and `range_mut` methods.
r? `@Mark-Simulacrum`
Auto merge of #83529 - richkadel:demangler, r=tmandry
Make rust-demangler installable
Adds bootstrap rules to support installing rust-demangler, as an optional, in-tree `extended` tool. It can be included by updating `config.toml`, setting `extended = true`, and then either (a) adding `"rust-demangler"` to the `tools` array, or by enabling `profiler = true`. In other words, it is a _default_ `extended` tool if `profiler = true`.
When compiling with `-Z instrument-coverage`, the coverage reports are
generated by `llvm-cov`. `llvm-cov` includes a built-in demangler for
C++, and an option to supply an alternate demangler. For Rust, we have
`rust-demangler`, currently used in `rustc` coverage tests.
Fuchsia's toolchain for Rust is built via `./x.py install`. Fuchsia is
adding support for Rust coverage, and we need to include the
`rust-demangler` in the installed `bin` directory.
Auto merge of #81507 - weiznich:add_diesel_to_cargo_test, r=Mark-Simulacrum
Adding diesel to the cargotest suite
As discussed in https://github.com/rust-lang/rust/issues/79560#issuecomment-767542364 this adds diesel to the compilers test suite. This is basically a reopened version of #79599, but now with the backing of the compiler team.
In the codegen and codegen-units test suites, the `//` comment markers
were kept in order not to affect any source locations. This is because
these tests cannot be automatically `--bless`ed.
Rich Kadel [Fri, 26 Mar 2021 20:02:46 +0000 (13:02 -0700)]
Make rust-demangler installable
Adds bootstrap rules to support installing rust-demangler.
When compiling with `-Z instrument-coverage`, the coverage reports are
generated by `llvm-cov`. `llvm-cov` includes a built-in demangler for
C++, and an option to supply an alternate demangler. For Rust, we have
`rust-demangler`, currently used in `rustc` coverage tests.
Fuchsia's toolchain for Rust is built via `./x.py install`. Fuchsia is
adding support for Rust coverage, and we need to include the
`rust-demangler` in the installed `bin` directory.
Configured rust-demangler as an in-tree extended tool.
Added tests to support `./x.py test rust-demangler`.
Install with extended tools by default only if `profiler = true`.
Rollup merge of #83804 - detrumi:build-type-ir-on-stable, r=petrochenkov
Remove nightly features in rustc_type_ir
`rustc_type_ir` will be used as a type library by Chalk, which we want to be able to build on stable, so this PR removes the current nightly features used.
Rollup merge of #83787 - digama0:patch-1, r=bjorn3
Monomorphization doc fix
Only public items are monomorphization roots. This can be confirmed by noting that this program compiles:
```rust
fn foo<T>() { if true { foo::<Option<T>>() } }
fn bar() { foo::<()>() }
```
See also the [zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Why.20are.20non.20public.20items.20monomorphization.20roots.3F).
I think it makes sense to uplift this guideline (in a milder form) into
std docs. Printing and producing errors is something that even
non-expert users do frequently, so it is useful to give at least some
indication of what a typical error message looks like.
Auto merge of #83738 - jyn514:only-load-some-crates, r=petrochenkov
rustdoc: Don't load all extern crates unconditionally
Instead, only load the crates that are linked to with intra-doc links.
This doesn't help very much with any of rustdoc's fundamental issues
with freezing the resolver, but it at least fixes a stable-to-stable
regression, and makes the crate loading model somewhat more consistent
with rustc's. I tested and it unfortunately does not help at all with https://github.com/rust-lang/rust/pull/82496.
Closes https://github.com/rust-lang/rust/issues/68427. Let me know if you want me to open a separate issue for not freezing the resolver.
r? `@petrochenkov` cc `@eddyb` `@ollie27`
projects / rust.git / log