Introduce `ChunkedBitSet` and use it for some dataflow analyses.
This reduces peak memory usage significantly for some programs with very
large functions, such as:
- `keccak`, `unicode_normalization`, and `match-stress-enum`, from
the `rustc-perf` benchmark suite;
- `http-0.2.6` from crates.io.
The new type is used in the analyses where the bitsets can get huge
(e.g. 10s of thousands of bits): `MaybeInitializedPlaces`,
`MaybeUninitializedPlaces`, and `EverInitializedPlaces`.
Some refactoring was required in `rustc_mir_dataflow`. All existing
analysis domains are either `BitSet` or a trivial wrapper around
`BitSet`, and access in a few places is done via `Borrow<BitSet>` or
`BorrowMut<BitSet>`. Now that some of these domains are `ClusterBitSet`,
that no longer works. So this commit replaces the `Borrow`/`BorrowMut`
usage with a new trait `BitSetExt` containing the needed bitset
operations. The impls just forward these to the underlying bitset type.
This required fiddling with trait bounds in a few places.
The commit also:
- Moves `static_assert_size` from `rustc_data_structures` to
`rustc_index` so it can be used in the latter; the former now
re-exports it so existing users are unaffected.
- Factors out some common "clear excess bits in the final word"
functionality in `bit_set.rs`.
- Uses `fill` in a few places instead of loops.
bors [Tue, 22 Feb 2022 20:50:38 +0000 (20:50 +0000)]
Auto merge of #83706 - a1phyr:fix_vec_layout_calculation, r=JohnTitor
Fix a layout possible miscalculation in `alloc::RawVec`
A layout miscalculation could happen in `RawVec` when used with a type whose size isn't a multiple of its alignment. I don't know if such type can exist in Rust, but the Layout API provides ways to manipulate such types. Anyway, it is better to calculate memory size in a consistent way.
bors [Tue, 22 Feb 2022 14:41:26 +0000 (14:41 +0000)]
Auto merge of #94254 - matthiaskrgr:rollup-7llbjhd, r=matthiaskrgr
Rollup of 7 pull requests
Successful merges:
- #94169 (Fix several asm! related issues)
- #94178 (tidy: fire less "ignoring file length unneccessarily" warnings)
- #94179 (solarish current_exe using libc call directly)
- #94196 (compiletest: Print process output info with less whitespace)
- #94208 (Add the let else tests found missing in the stabilization report)
- #94237 (Do not suggest wrapping an item if it has ambiguous un-imported methods)
- #94246 (ScalarMaybeUninit is explicitly hexadecimal in its formatting)
Matthias Krüger [Tue, 22 Feb 2022 11:16:33 +0000 (12:16 +0100)]
Rollup merge of #94237 - compiler-errors:dont-wrap-ambiguous-receivers, r=lcnr
Do not suggest wrapping an item if it has ambiguous un-imported methods
If the method is defined for the receiver we have, but is ambiguous during probe, then it probably comes from one of several traits that just weren't `use`d. Don't suggest wrapping the receiver in `Box`/etc., even if that makes the method probe unambiguous.
------------------------------------------
stderr:
------------------------------------------
Invalid command: Tried to use the previous path in the first command on line 10
Error: "Jsondocck failed for /data/ne321/rust/src/test/rustdoc-json/traits/supertrait.rs"
error: jsondocck failed!
status: exit status: 1
command: "/data/ne321/rust/build/x86_64-unknown-linux-gnu/stage0-tools-bin/jsondocck" "--doc-dir" "/data/ne321/rust/build/x86_64-unknown-linux-gnu/test/rustdoc-json/traits/supertrait" "--template" "/data/ne321/rust/src/test/rustdoc-json/traits/supertrait.rs"
stdout: none
--- stderr -------------------------------
Invalid command: Tried to use the previous path in the first command on line 10
Error: "Jsondocck failed for /data/ne321/rust/src/test/rustdoc-json/traits/supertrait.rs"
------------------------------------------
Matthias Krüger [Tue, 22 Feb 2022 11:16:29 +0000 (12:16 +0100)]
Rollup merge of #94178 - est31:tolerant_lines_check, r=Mark-Simulacrum
tidy: fire less "ignoring file length unneccessarily" warnings
This avoids a situation where a file is at the border of the limit,
and alternates between hitting the limit and not hitting it, causing
a back and forth of addition of the ignore-tidy-linelength directive.
As an example, consider the ignore-tidy-filelength of compiler/rustc_typeck/src/collect.rs.
Matthias Krüger [Tue, 22 Feb 2022 11:16:28 +0000 (12:16 +0100)]
Rollup merge of #94169 - Amanieu:asm_stuff, r=nagisa
Fix several asm! related issues
This is a combination of several fixes, each split into a separate commit. Splitting these into PRs is not practical since they conflict with each other.
Matthias Krüger [Tue, 22 Feb 2022 11:16:28 +0000 (12:16 +0100)]
Rollup merge of #94169 - Amanieu:asm_stuff, r=nagisa
Fix several asm! related issues
This is a combination of several fixes, each split into a separate commit. Splitting these into PRs is not practical since they conflict with each other.
bors [Tue, 22 Feb 2022 07:54:22 +0000 (07:54 +0000)]
Auto merge of #93839 - Mark-Simulacrum:delete-json-rust-deserialization, r=nnethercote
Simplify rustc_serialize by dropping support for decoding into JSON
This PR currently bundles two (somewhat separate) tasks.
First, it removes the JSON Decoder trait impl, which permitted going from JSON to Rust structs. For now, we keep supporting JSON deserialization, but only to `Json` (an equivalent of serde_json::Value). The primary hard to remove user there is for custom targets -- which need some form of JSON deserialization -- but they already have a custom ad-hoc pass for moving from Json to a Rust struct.
A [comment](https://github.com/rust-lang/rust/blob/e7aca895980f25f6d2d3c48e10fd04656764d1e4/compiler/rustc_target/src/spec/mod.rs#L1653) there suggests that it would be impractical to move them to a Decodable-based impl, at least without backwards compatibility concerns. I suspect that if we were widely breaking compat there, it would make sense to use serde_json at this point which would produce better error messages; the types in rustc_target are relatively isolated so we would not particularly suffer from using serde_derive.
The second part of the PR (all but the first commit) is to simplify the Decoder API by removing the non-primitive `read_*` functions. These primarily add indirection (through a closure), which doesn't directly cause a performance issue (the unique closure types essentially guarantee monomorphization), but does increase the amount of work rustc and LLVM need to do. This could be split out to a separate PR, but is included here in part to help motivate the first part.
Future work might consist of:
* Specializing enum discriminant encoding to avoid leb128 for small enums (since we know the variant count, we can directly use read/write u8 in almost all cases)
* Adding new methods to support faster deserialization (e.g., access to the underlying byte stream)
* Currently these are somewhat ad-hoc supported by specializations for e.g. `Vec<u8>`, but other types which could benefit don't today.
* Removing the Decoder trait entirely in favor of a concrete type -- today, we only really have one impl of it modulo wrappers used for specialization-based dispatch.
Highly recommend review with whitespace changes off, as the removal of closures frequently causes things to be de-indented.
bors [Mon, 21 Feb 2022 22:53:45 +0000 (22:53 +0000)]
Auto merge of #94225 - matthiaskrgr:rollup-0728x8n, r=matthiaskrgr
Rollup of 10 pull requests
Successful merges:
- #91192 (Some improvements to the async docs)
- #94143 (rustc_const_eval: adopt let else in more places)
- #94156 (Gracefully handle non-UTF-8 string slices when pretty printing)
- #94186 (Update pin_static_ref stabilization version.)
- #94189 (Implement LowerHex on Scalar to clean up their display in rustdoc)
- #94190 (Use Metadata::modified instead of FileTime::from_last_modification_ti…)
- #94203 (CTFE engine: Scalar: expose size-generic to_(u)int methods)
- #94211 (Better error if the user tries to do assignment ... else)
- #94215 (trait system: comments and small nonfunctional changes)
- #94220 (Correctly handle miniz_oxide extern crate declaration)
Matthias Krüger [Mon, 21 Feb 2022 18:36:51 +0000 (19:36 +0100)]
Rollup merge of #94190 - bjorn3:less_filetime, r=Mark-Simulacrum
Use Metadata::modified instead of FileTime::from_last_modification_ti…
…me in run_cargo
Metadata::modified works in all platforms supported by the filetime
crate. This changes brings rustbuild a tiny bit closer towards dropping
the filetime dependency.
Matthias Krüger [Mon, 21 Feb 2022 18:36:48 +0000 (19:36 +0100)]
Rollup merge of #94143 - est31:let_else_const_eval, r=lcnr
rustc_const_eval: adopt let else in more places
Continuation of #89933, #91018, #91481, #93046, #93590, #94011.
I have extended my clippy lint to also recognize tuple passing and match statements. The diff caused by fixing it is way above 1 thousand lines. Thus, I split it up into multiple pull requests to make reviewing easier. This PR handles rustc_const_eval.
Amanieu d'Antras [Thu, 17 Feb 2022 14:16:52 +0000 (14:16 +0000)]
On ARM, use relocation_model to detect whether r9 should be reserved
The previous approach of checking for the reserve-r9 target feature
didn't actually work because LLVM only sets this feature very late when
initializing the per-function subtarget.
Amanieu d'Antras [Thu, 17 Feb 2022 14:16:52 +0000 (14:16 +0000)]
On ARM, use relocation_model to detect whether r9 should be reserved
The previous approach of checking for the reserve-r9 target feature
didn't actually work because LLVM only sets this feature very late when
initializing the per-function subtarget.
This PR has 3 relevant steps which are is split in distinct commits.
The first commit now interns `List<Ty<'tcx>>` and `List<GenericArg<'tcx>>` together, potentially reusing memory while allowing free conversions between these two using `List<Ty<'tcx>>::as_substs()` and `SubstsRef<'tcx>::try_as_type_list()`.
Using this, we then use `&'tcx List<Ty<'tcx>>` instead of a `SubstsRef<'tcx>` for tuple fields, simplifying a bunch of code.
Finally, as tuple fields and other generic arguments now use a different `TypeFoldable<'tcx>` impl, we optimize the impl for `List<Ty<'tcx>>` improving perf by slightly less than 1% in tuple heavy benchmarks.
This PR was likely responsible for a relatively large regression in
dist-x86_64-msvc-alt builder times, from approximately 1.7 to 2.8 hours,
bringing that builder into the pool of the slowest builders we currently have.
This seems to be limited to the alt builder due to needing parallel-compiler
enabled, likely leading to slow LLVM compilation for some reason. See some
investigation in [this Zulip stream](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/msvc.28.3F.29.20builders.20running.20much.20slower).
cc `@lcnr` `@oli-obk` `@b-naber` (per original PRs review/author)
We can re-apply this PR once the regression is fixed, but it is sufficiently large that I don't think keeping this on master is viable in the meantime unless there's a very strong case to be made for it. Alternatively, we can disable that builder (it's not critical since it's an alt build), but that obviously carries its own costs.
bors [Mon, 21 Feb 2022 10:06:24 +0000 (10:06 +0000)]
Auto merge of #94108 - compiler-errors:just-confirmation-normalization, r=jackh726
Normalize obligation and expected trait_refs in confirm_poly_trait_refs
Consolidate normalization the obligation and expected trait refs in `confirm_poly_trait_refs`. Also, _always_ normalize these trait refs -- we were already normalizing the obligation trait ref when confirming closure and generator candidates, but this does it for fn pointer confirmation as well.
This presumably does more work in the case that the obligation's trait ref is already normalized, but we can see from the perf runs in #94070, it actually (paradoxically, perhaps) improves performance when paired with logic that normalizes projections in fulfillment loop.
Scott McMurray [Mon, 21 Feb 2022 07:25:18 +0000 (23:25 -0800)]
Stop manually SIMDing in swap_nonoverlapping
Like I previously did for `reverse`, this leaves it to LLVM to pick how to vectorize it, since it can know better the chunk size to use, compared to the "32 bytes always" approach we currently have.
It does still need logic to type-erase where appropriate, though, as while LLVM is now smart enough to vectorize over slices of things like `[u8; 4]`, it fails to do so over slices of `[u8; 3]`.
As a bonus, this also means one no longer gets the spurious `memcpy`(s?) at the end up swapping a slice of `__m256`s: <https://rust.godbolt.org/z/joofr4v8Y>
est31 [Mon, 21 Feb 2022 03:38:39 +0000 (04:38 +0100)]
Expand let-else allow tests
The #[allow(...)] directive was tested for the body and the pattern,
but non-presence of it wasn't tested. Furthermore, it wasn't tested
for the expression. We add expression tests as well as ones checking
the non-presence of the directive.
This PR was likely responsible for a relatively large regression in
dist-x86_64-msvc-alt builder times, from approximately 1.7 to 2.8 hours,
bringing that builder into the pool of the slowest builders we currently have.
This seems to be limited to the alt builder due to needing parallel-compiler
enabled, likely leading to slow LLVM compilation for some reason.
Mark Rousskov [Wed, 9 Feb 2022 22:05:44 +0000 (17:05 -0500)]
Remove support for JSON deserialization to Rust
This is no longer used by the compiler itself, and removing this support opens
the door to massively simplifying the Decodable/Decoder API by dropping the
self-describing deserialization support (necessary for JSON).
------------------------------------------
stderr:
------------------------------------------
Invalid command: Tried to use the previous path in the first command on line 10
Error: "Jsondocck failed for /data/ne321/rust/src/test/rustdoc-json/traits/supertrait.rs"
error: jsondocck failed!
status: exit status: 1
command: "/data/ne321/rust/build/x86_64-unknown-linux-gnu/stage0-tools-bin/jsondocck" "--doc-dir" "/data/ne321/rust/build/x86_64-unknown-linux-gnu/test/rustdoc-json/traits/supertrait" "--template" "/data/ne321/rust/src/test/rustdoc-json/traits/supertrait.rs"
stdout: none
--- stderr -------------------------------
Invalid command: Tried to use the previous path in the first command on line 10
Error: "Jsondocck failed for /data/ne321/rust/src/test/rustdoc-json/traits/supertrait.rs"
------------------------------------------
Frank Steffahn [Thu, 3 Feb 2022 21:16:06 +0000 (22:16 +0100)]
Improve `unused_unsafe` lint
Main motivation: Fixes some issues with the current behavior. This PR is
more-or-less completely re-implementing the unused_unsafe lint; it’s also only
done in the MIR-version of the lint, the set of tests for the `-Zthir-unsafeck`
version no longer succeeds (and is thus disabled, see `lint-unused-unsafe.rs`).
doesn’t create any warnings. This situation is not unrealistic to come by, the
inner `unsafe` block could e.g. come from a macro. Actually, this PR even
includes removal of one unused `unsafe` in the standard library that was missed
in a similar situation. (The inner `unsafe` coming from an external macro hides
the warning, too.)
The reason behind this problem is how the check currently works:
* While generating MIR, it already skips nested unsafe blocks (i.e. unsafe
nested in other unsafe) so that the inner one is always the one considered
unused
* To differentiate the cases of no unsafe operations inside the `unsafe` vs.
a surrounding `unsafe` block, there’s some ad-hoc magic walking up the HIR to
look for surrounding used `unsafe` blocks.
There’s a lot of problems with this approach besides the one presented above.
E.g. the MIR-building uses checks for `unsafe_op_in_unsafe_fn` lint to decide
early whether or not `unsafe` blocks in an `unsafe fn` are redundant and ought
to be removed.
```rs
unsafe fn granular_disallow_op_in_unsafe_fn() {
unsafe {
#[deny(unsafe_op_in_unsafe_fn)]
{
unsf();
}
}
}
```
```
error: call to unsafe function is unsafe and requires unsafe block (error E0133)
--> src/main.rs:13:13
|
13 | unsf();
| ^^^^^^ call to unsafe function
|
note: the lint level is defined here
--> src/main.rs:11:16
|
11 | #[deny(unsafe_op_in_unsafe_fn)]
| ^^^^^^^^^^^^^^^^^^^^^^
= note: consult the function's documentation for information on how to avoid undefined behavior
warning: unnecessary `unsafe` block
--> src/main.rs:10:5
|
9 | unsafe fn granular_disallow_op_in_unsafe_fn() {
| --------------------------------------------- because it's nested under this `unsafe` fn
10 | unsafe {
| ^^^^^^ unnecessary `unsafe` block
|
= note: `#[warn(unused_unsafe)]` on by default
```
Here, the intermediate `unsafe` was ignored, even though it contains a unsafe
operation that is not allowed to happen in an `unsafe fn` without an additional `unsafe` block.
Also closures were problematic and the workaround/algorithms used on current
nightly didn’t work properly. (I skipped trying to fully understand what it was
supposed to do, because this PR uses a completely different approach.)
```rs
fn nested() {
unsafe {
unsafe { unsf() }
}
}
```
```
warning: unnecessary `unsafe` block
--> src/main.rs:10:9
|
9 | unsafe {
| ------ because it's nested under this `unsafe` block
10 | unsafe { unsf() }
| ^^^^^^ unnecessary `unsafe` block
|
= note: `#[warn(unused_unsafe)]` on by default
```
vs
```rs
fn nested() {
let _ = || unsafe {
let _ = || unsafe { unsf() };
};
}
```
```
warning: unnecessary `unsafe` block
--> src/main.rs:9:16
|
9 | let _ = || unsafe {
| ^^^^^^ unnecessary `unsafe` block
|
= note: `#[warn(unused_unsafe)]` on by default
I also dislike the fact that it always suggests keeping the outermost `unsafe`.
E.g. for
```rs
fn granularity() {
unsafe {
unsafe { unsf() }
unsafe { unsf() }
unsafe { unsf() }
}
}
```
I prefer if `rustc` suggests removing the more-course outer-level `unsafe`
instead of the fine-grained inner `unsafe` blocks, which it currently does on nightly:
```
warning: unnecessary `unsafe` block
--> src/main.rs:10:9
|
9 | unsafe {
| ------ because it's nested under this `unsafe` block
10 | unsafe { unsf() }
| ^^^^^^ unnecessary `unsafe` block
|
= note: `#[warn(unused_unsafe)]` on by default
Needless to say, this PR addresses all these points. For context, as far as my
understanding goes, the main advantage of skipping inner unsafe blocks was that
a test case like
```rs
fn top_level_used() {
unsafe {
unsf();
unsafe { unsf() }
unsafe { unsf() }
unsafe { unsf() }
}
}
```
should generate some warning because there’s redundant nested `unsafe`, however
every single `unsafe` block _does_ contain some statement that uses it. Of course
this PR doesn’t aim change the warnings on this kind of code example, because
the current behavior, warning on all the inner `unsafe` blocks, makes sense in this case.
As mentioned, during MIR building all the unsafe blocks *are* kept now, and usage
is attributed to them. The way to still generate a warning like
```
warning: unnecessary `unsafe` block
--> src/main.rs:11:9
|
9 | unsafe {
| ------ because it's nested under this `unsafe` block
10 | unsf();
11 | unsafe { unsf() }
| ^^^^^^ unnecessary `unsafe` block
|
= note: `#[warn(unused_unsafe)]` on by default
in this case is by emitting a `unused_unsafe` warning for all of the `unsafe`
blocks that are _within a **used** unsafe block_.
The previous code had a little HIR traversal already anyways to collect a set of
all the unsafe blocks (in order to afterwards determine which ones are unused
afterwards). This PR uses such a traversal to do additional things including logic
like _always_ warn for an `unsafe` block that’s inside of another **used**
unsafe block. The traversal is expanded to include nested closures in the same go,
this simplifies a lot of things.
The whole logic around `unsafe_op_in_unsafe_fn` is a little complicated, there’s
some test cases of corner-cases in this PR. (The implementation involves
differentiating between whether a used unsafe block was used exclusively by
operations where `allow(unsafe_op_in_unsafe_fn)` was active.) The main goal was
to make sure that code should compile successfully if all the `unused_unsafe`-warnings
are addressed _simultaneously_ (by removing the respective `unsafe` blocks)
no matter how complicated the patterns of `unsafe_op_in_unsafe_fn` being
disallowed and allowed throughout the function are.
One noteworthy design decision I took here: An `unsafe` block
with `allow(unused_unsafe)` **is considered used** for the purposes of
linting about redundant contained unsafe blocks. So while
```rs
projects / rust.git / log