Auto merge of #88161 - michaelwoerister:fix-whole-archive-no-bundle, r=petrochenkov
Fix handling of +whole-archive native link modifier.
This PR fixes a bug in `add_upstream_native_libraries` that led to the `+whole-archive` modifier being ignored when linking in native libs.
~~Note that the PR does not address the situation when `+whole-archive` is combined with `+bundle`.~~
`@wesleywiser's` commit adds validation code that turns combining `+whole-archive` with `+bundle` into an error.
Auto merge of #88448 - xu-cheng:btree-blk-build, r=Mark-Simulacrum
BTreeMap/BTreeSet::from_iter: use bulk building to improve the performance
Bulk building is a common technique to increase the performance of building a fresh btree map. Instead of inserting items one-by-one, we sort all the items beforehand then create the BtreeMap in bulk.
Auto merge of #83214 - cjgillot:dep-map, r=michaelwoerister
Mmap the incremental data instead of reading it.
Instead of reading the full incremental state using `fs::read_file`, we memmap it using a private read-only file-backed map.
This allows the system to reclaim any memory we are not using, while ensuring we are not polluted by
outside modifications to the file.
Suggested in https://github.com/rust-lang/rust/pull/83036#issuecomment-800458082 by `@bjorn3`
Auto merge of #88362 - pietroalbini:bump-stage0, r=Mark-Simulacrum
Pin bootstrap checksums and add a tool to update it automatically
:warning: :warning: This is just a proactive hardening we're performing on the build system, and it's not prompted by any known compromise. If you're aware of security issues being exploited please [check out our responsible disclosure page](https://www.rust-lang.org/policies/security). :warning: :warning:
---
This PR aims to improve Rust's supply chain security by pinning the checksums of the bootstrap compiler downloaded by `x.py`, preventing a compromised `static.rust-lang.org` from affecting building the compiler. The checksums are stored in `src/stage0.json`, which replaces `src/stage0.txt`. This PR also adds a tool to automatically update the bootstrap compiler.
The changes in this PR were originally discussed in [Zulip](https://zulip-archive.rust-lang.org/stream/241545-t-release/topic/pinning.20stage0.20hashes.html).
## Potential attack
Before this PR, an attacker who wanted to compromise the bootstrap compiler would "just" need to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries and corresponding `.sha256` files to `static.rust-lang.org`.
There is no signature verification in `x.py` as we don't want the build system to depend on GPG. Also, since the checksums were not pinned inside the repository, they were downloaded from `static.rust-lang.org` too: this only protected from accidental changes in `static.rust-lang.org` that didn't change the `*.sha256` files. The attack would allow the attacker to compromise past and future invocations of `x.py`.
## Mitigations introduced in this PR
This PR adds pinned checksums for all the bootstrap components in `src/stage0.json` instead of downloading the checksums from `static.rust-lang.org`. This changes the attack scenario to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries to `static.rust-lang.org`.
3. Land a (reviewed) change in the `rust-lang/rust` repository changing the pinned hashes.
Even with a successful attack, existing clones of the Rust repository won't be affected, and once the attack is detected reverting the pinned hashes changes should be enough to be protected from the attack. This also enables further mitigations to be implemented in following PRs, such as verifying signatures when pinning new checksums (removing the trust on first use aspect of this PR) and adding a check in CI making sure a PR updating the checksum has not been tampered with (see the future improvements section).
## Additional changes
There are additional changes implemented in this PR to enable the mitigation:
* The `src/stage0.txt` file has been replaced with `src/stage0.json`. The reasoning for the change is that there is existing tooling to read and manipulate JSON files compared to the custom format we were using before, and the slight challenge of manually editing JSON files (no comments, no trailing commas) are not a problem thanks to the new `bump-stage0`.
* A new tool has been added to the repository, `bump-stage0`. When invoked, the tool automatically calculates which release should be used as the bootstrap compiler given the current version and channel, gathers all the relevant checksums and updates `src/stage0.json`. The tool can be invoked by running:
```
./x.py run src/tools/bump-stage0
```
* Support for downloading releases from `https://dev-static.rust-lang.org` has been removed, as it's not possible to verify checksums there (it's customary to replace existing artifacts there if a rebuild is warranted). This will require a change to the release process to avoid bumping the bootstrap compiler on beta before the stable release.
## Future improvements
* Add signature verification as part of `bump-stage0`, which would require the attacker to also obtain the release signing keys in order to successfully compromise the bootstrap compiler. This would be fine to add now, as the burden of installing the tool to verify signatures would only be placed on whoever updates the bootstrap compiler, instead of everyone compiling Rust.
* Add a check on CI that ensures the checksums in `src/stage0.json` are the expected ones. If a PR changes the stage0 file CI should also run the `bump-stage0` tool and fail if the output in CI doesn't match the committed file. This prevents the PR author from tweaking the output of the tool manually, which would otherwise be close to impossible for a human to detect.
* Automate creating the PRs bumping the bootstrap compiler, by setting up a scheduled job in GitHub Actions that runs the tool and opens a PR.
* Investigate whether a similar mitigation can be done for "download from CI" components like the prebuilt LLVM.
Auto merge of #88678 - matthewjasper:if-boolean-scoping, r=oli-obk
Change scope of temporaries in match guards
Each pattern in a match arm has its own copy of the match guard in MIR, with its own temporary, so it has to be dropped before the the guards are joined to the single copy of the arm. This PR changes `then_else_break` to allow it to put the temporary in the innermost scope possible. This change isn't done for `if` expressions because that affects a large number of mir-opt tests and could more significantly affect performance.
there is also a random test in here about array repeat expressions that I already had on this branch but it seems to fit the theme of this PR so kept it...
Auto merge of #88631 - camelid:sugg-span, r=davidtwco
Improve structured tuple struct suggestion
Previously, the span was just for the constructor name, which meant it
would result in syntactically-invalid code when applied. Now, the span
is for the entire expression.
I also changed it to use `span_suggestion_verbose`, for two reasons:
1. Now that the suggestion span has been corrected, the output is a bit
cluttered and hard to read. Putting the suggestion its own window
creates more space.
2. It's easier to see what's being suggested, since now the version
after the suggestion is applied is shown.
Auto merge of #88435 - cjgillot:no-walk-crate, r=Aaron1011
Avoid invoking the hir_crate query to traverse the HIR
Walking the HIR tree is done using the `hir_crate` query. However, this is unnecessary, since `hir_owner(CRATE_DEF_ID)` provides the same information. Since depending on `hir_crate` forces dependents to always be executed, this leads to unnecessary work.
By splitting HIR and attributes visits, we can avoid an edge to `hir_crate` when trying to visit the HIR tree.
Stop allocating vtable entries for non-object-safe methods
Current a vtable entry is allocated for all associated fns, even if the method is not object-safe: https://godbolt.org/z/h7vx6f35T
As a result, each vtable for `Iterator`' currently consumes 74 `usize`s. This PR stops allocating vtable entries for those methods, reducing vtable size of each `Iterator` vtable to 7 `usize`s.
Note that this PR introduces will cause more invocations of `is_vtable_safe_method`. So a perf run might be needed. If result isn't favorable then we might need to query-ify `is_vtable_safe_method`.
Matthew Jasper [Sun, 5 Sep 2021 17:50:55 +0000 (18:50 +0100)]
Change scope of temporaries in match guards
Each pattern in a match arm has its own copy of the match guard in MIR,
with its own temporary, so it has to be dropped before the the guards
are joined to the single copy of the arm.
Provide `layout_of` automatically (given tcx + param_env + error handling).
After #88337, there's no longer any uses of `LayoutOf` within `rustc_target` itself, so I realized I could move the trait to `rustc_middle::ty::layout` and redesign it a bit.
This is similar to #88338 (and supersedes it), but at no ergonomic loss, since there's no funky `C: LayoutOf<Ty = Ty>` -> `Ty: TyAbiInterface<C>` generic `impl` chain, and each `LayoutOf` still corresponds to one `impl` (of `LayoutOfHelpers`) for the specific context.
After this PR, this is what's needed to get `trait LayoutOf` (with the `layout_of` method) implemented on some context type:
* `TyCtxt`, via `HasTyCtxt`
* `ParamEnv`, via `HasParamEnv`
* a way to transform `LayoutError`s into the desired error type
* an error type of `!` can be paired with having `cx.layout_of(...)` return `TyAndLayout` *without* `Result<...>` around it, such as used by codegen
* this is done through a new `LayoutOfHelpers` trait (and so is specifying the type of `cx.layout_of(...)`)
When going through this path (and not bypassing it with a manual `impl` of `LayoutOf`), the end result is that only the error case can be customized, the query itself and the success paths are guaranteed to be uniform.
(**EDIT**: just noticed that because of the supertrait relationship, you cannot actually implement `LayoutOf` yourself, the blanket `impl` fully covers all possible context types that could ever implement it)
Part of the motivation for this shape of API is that I've been working on querifying `FnAbi::of_*`, and what I want/need to introduce for that looks a lot like the setup in this PR - in particular, it's harder to express the `FnAbi` methods in `rustc_target`, since they're much more tied to `rustc` concepts.
Change return type for T::{log,log2,log10} to u32. The value is at
most 128, and this is consistent with using u32 for small values
elsewhere (e.g. BITS, count_ones, leading_zeros).
Auto merge of #88604 - camelid:rustdoc-lifetime-bounds, r=GuillaumeGomez
rustdoc: Clean up handling of lifetime bounds
Previously, rustdoc recorded lifetime bounds by rendering them into the
name of the lifetime parameter. Now, it leaves the name as the actual
name and instead records lifetime bounds in an `outlives` list, similar
to how type parameter bounds are recorded.
Also, higher-ranked lifetimes cannot currently have bounds, so I simplified
the code to reflect that.
Auto merge of #88662 - m-ou-se:rollup-h6lgp7k, r=m-ou-se
Rollup of 4 pull requests
Successful merges:
- #88257 (Provide more context on incorrect inner attribute)
- #88432 (Fix a typo in raw_vec)
- #88511 (x.py clippy: don't run with --all-targets by default)
- #88657 (Fix 2021 `dyn` suggestion that used code as label)
Auto merge of #88469 - patrick-gu:master, r=dtolnay
Add links in docs for some primitive types
This pull request adds additional links in existing documentation of some of the primitive types.
Where items are linked only once, I have used the `[link](destination)` format. For items in `std`, I have linked directly to the HTML, since although the primitives are in `core`, they are not displayed on `core` documentation. I was unsure of what length I should keep lines of documentation to, so I tried to keep them within reason.
Additionally, I have avoided excessively linking to keywords like `self` when they are not relevant to the documentation. I can add these links if it would be an improvement.
I hope this can improve Rust. Please let me know if there's anything I did wrong!
Auto merge of #88364 - pietroalbini:llvm-install-filecheck, r=Mark-Simulacrum
Make sure FileCheck is copied in the LLVM output directory
The tool, which is needed by parts of our test suite, is built as part of LLVM but is *not* copied to the directory containing the output LLVM binaries. This adds a flag to ensure the binary is copied. This shouldn't add any extra built time, as the flag just installs extra binaries that were already compiled.
This is not strictly needed for the test suite to work (as it also checks `build/$target/llvm/build/bin` for the binary), but it allows deleting the `build/$TARGET/llvm/build` directory (which also contains the intermediary build artifacts) without affecting the test suite, saving disk space.
Auto merge of #88550 - dpaoliello:dpaoliello/allocdebuginfo, r=estebank
Include debug info for the allocator shim
Issue Details:
In some cases it is necessary to generate an "allocator shim" to forward various Rust allocation functions (e.g., `__rust_alloc`) to an underlying function (e.g., `malloc`). However, since this allocator shim is a manually created LLVM module it is not processed via the normal module processing code and so no debug info is generated for it (if debugging info is enabled).
Fix Details:
* Modify the `debuginfo` code to allow creating debug info for a module without a `CodegenCx` (since it is difficult, and expensive, to create one just to emit some debug info).
* After creating the allocator shim add in basic debug info.
Auto merge of #88538 - bjorn3:no_session_in_crate_loader, r=petrochenkov
CrateLocator refactorings
This makes the `CrateLocator` a lot cleaner IMHO and much more self-contained. The last commit removes `extra_filename` from the crate metadata. This is an **insta-stable** change as it allows a crate like `libfoo-abc.rlib` to be used as dependency and then be renamed as `libfoo-bcd.rlib` while still being found as indirect dependency. This may reduce performance when there are a lot of versions of the same crate available as the extra filename won't be used to do an early rejection of crates before trying to load metadata, but it makes the logic to find the right filename a lot cleaner.
Auto merge of #88482 - athei:add-import-test, r=the8472
Add regression test for a spurious import
This PR adds a test that verifies that the bug described in the linked issue does not creep back into the code. In essence it checks that compiling some specific code (that uses 128 bit multiplication) with a specific set of compiler options does not lead to a spurious import of a panic function.
I noticed that other wasm tests use `# only-wasm32-bare` in their `Makefile`. This will skip the test for me. I did not find out how to run this test locally. Maybe someone can help.
Auto merge of #88572 - matthewjasper:if-let-scoping-fix, r=oli-obk
Fix drop handling for `if let` expressions
MIR lowering for `if let` expressions is now more complicated now that
`if let` exists in HIR. This PR adds a scope for the variables bound in
an `if let` expression and then uses an approach similar to how we
handle loops to ensure that we reliably drop the correct variables.
Closes #88307
cc `@flip1995` `@richkadel` `@c410-f3r`
Auto merge of #88618 - m-ou-se:rollup-6tss5z6, r=m-ou-se
Rollup of 7 pull requests
Successful merges:
- #88202 (Add an example for deriving PartialOrd on enums)
- #88483 (Fix LLVM libunwind build for non-musl targets)
- #88507 (Add test case for using `slice::fill` with MaybeUninit)
- #88557 (small const generics cleanup)
- #88579 (remove redundant / misplaced sentence from docs)
- #88610 (Update outdated docs of array::IntoIter::new.)
- #88613 (Update primitive docs for rust 2021.)
Mara Bos [Fri, 3 Sep 2021 11:30:49 +0000 (13:30 +0200)]
Rollup merge of #88579 - ast-ral:master, r=m-ou-se
remove redundant / misplaced sentence from docs
Removes sentence that seems to have drifted down into the examples section. Luckily, someone already added an explanation of what happens with packed structs back into the initial section of the doc entry and this wayward sentence can likely just be deleted.
Mara Bos [Fri, 3 Sep 2021 11:30:46 +0000 (13:30 +0200)]
Rollup merge of #88483 - jethrogb:jb/llvm-libunwind-self-contained, r=petrochenkov
Fix LLVM libunwind build for non-musl targets
Broken in #85600. AFAICT, [only musl, mingw, and wasm](https://github.com/rust-lang/rust/blob/673d0db5e393e9c64897005b470bfeb6d5aec61b/compiler/rustc_target/src/spec/crt_objects.rs#L128-L132) should use the “self-contained” logic in rustbuild.
Mara Bos [Fri, 3 Sep 2021 11:30:46 +0000 (13:30 +0200)]
Rollup merge of #88202 - azdavis:master, r=jyn514
Add an example for deriving PartialOrd on enums
For some reason, I always forget which variants are smaller and which
are larger when you derive PartialOrd on an enum. And the wording in the
current docs is not entirely clear to me.
So, I often end up making a small enum, deriving PartialOrd on it, and
then writing a `#[test]` with an assert that the top one is smaller than
the bottom one (or the other way around) to figure out which way the
deriving goes.
So then I figured, it would be great if the standard library docs just
had that example, so if I keep forgetting, at least I can figure it out
quickly by looking at std's docs.
Auto merge of #88428 - petrochenkov:stmtid, r=Aaron1011
expand: Treat more macro calls as statement macro calls
This PR implements the suggestion from https://github.com/rust-lang/rust/pull/87981#issuecomment-906641052 and treats fn-like macro calls inside `StmtKind::Item` and `StmtKind::Semi` as statement macro calls, which is consistent with treatment of attribute invocations in the same positions and with token-based macro expansion model in general.
This also allows to remove a special case in `NodeId` assignment (previously tried in #87779), and to use statement `NodeId`s for linting (`assign_id!`).
Auto merge of #88386 - estebank:unmatched-delims, r=jackh726
Point at unclosed delimiters as part of the primary MultiSpan
Both the place where the parser encounters a needed closed delimiter and
the unclosed opening delimiter are important, so they should get the
same level of highlighting in the output.
Auto merge of #88363 - michaelwoerister:remapped-diagnostics, r=estebank
Path remapping: Make behavior of diagnostics output dependent on presence of --remap-path-prefix.
This PR fixes a regression (#87745) with `--remap-path-prefix` where the flag stopped causing diagnostic messages to be remapped as well. The regression was introduced in https://github.com/rust-lang/rust/pull/83813 where we erroneously assumed that remapping of diagnostic messages was not desired anymore (because #70642 partially undid that functionality with nobody objecting).
The issue is fixed by making `--remap-path-prefix` remap diagnostic messages again, including for paths that have been remapped in upstream crates (e.g. the standard library). This means that "sysroot-localization" (implemented in #70642) is also disabled if `rustc` is invoked with `--remap-path-prefix`. The assumption is that once someone starts explicitly remapping paths they also don't want paths to their local Rust installation in their build output.
In the future we might want to give more fine-grained control over this behavior via compiler flags (see https://github.com/rust-lang/rfcs/pull/3127 for a related RFC). For now this PR is intended as a regression fix.
This PR is an alternative to https://github.com/rust-lang/rust/pull/88191, which makes diagnostic messages be remapped unconditionally. That approach, however, would effectively revert #70642.
Noah Lev [Thu, 2 Sep 2021 16:54:32 +0000 (09:54 -0700)]
rustdoc: Higher-ranked lifetimes can't have bounds
This cleans up the other spot I found where rustdoc was rendering bounds
into the lifetime name itself. However, in this case, I don't think it
could have actually happened because higher-ranked lifetime definitions
aren't currently allowed to have bounds.
Noah Lev [Wed, 1 Sep 2021 23:20:14 +0000 (16:20 -0700)]
rustdoc: Clean up handling of lifetime bounds
Previously, rustdoc recorded lifetime bounds by rendering them into the
name of the lifetime parameter. Now, it leaves the name as the actual
name and instead records lifetime bounds in an `outlives` list, similar
to how type parameter bounds are recorded.
When we evaluate a projection predicate, we may produce sub-obligations. During trait evaluation, evaluating these sub-obligations might cause us to produce `EvaluatedToOkModuloRegions`.
When we cache the result of projection in our projection cache, we try to throw away some of the sub-obligations, so that we don't need to re-evaluate/process them the next time we need to perform this particular projection. However, we may end up throwing away predicates that will (recursively) evaluate to `EvaluatedToOkModuloRegions`. If we do, then the result of evaluating a predicate will depend on the state of the predicate cache - this is global untracked state, which interacts badly with incremental compilation.
To fix this, we now only discard global predicates that evaluate to `EvaluatedToOk`. This ensures that any predicates that (may) evaluate to `EvaluatedToOkModuloRegions` are kept in the cache, and influence the results of any queries which perform this projection.
Mara Bos [Thu, 2 Sep 2021 17:10:23 +0000 (19:10 +0200)]
Rollup merge of #88589 - xFrednet:00000-correct-comment-to-doc, r=petrochenkov
Correct doc comments inside `use_expr_visitor.rs`
Just a simple update. I haven't changed any content inside the comments, as they still seem correct. Have a wonderful rest of the day :upside_down_face:
projects / rust.git / log