Auto merge of #88362 - pietroalbini:bump-stage0, r=Mark-Simulacrum
Pin bootstrap checksums and add a tool to update it automatically
:warning: :warning: This is just a proactive hardening we're performing on the build system, and it's not prompted by any known compromise. If you're aware of security issues being exploited please [check out our responsible disclosure page](https://www.rust-lang.org/policies/security). :warning: :warning:
---
This PR aims to improve Rust's supply chain security by pinning the checksums of the bootstrap compiler downloaded by `x.py`, preventing a compromised `static.rust-lang.org` from affecting building the compiler. The checksums are stored in `src/stage0.json`, which replaces `src/stage0.txt`. This PR also adds a tool to automatically update the bootstrap compiler.
The changes in this PR were originally discussed in [Zulip](https://zulip-archive.rust-lang.org/stream/241545-t-release/topic/pinning.20stage0.20hashes.html).
## Potential attack
Before this PR, an attacker who wanted to compromise the bootstrap compiler would "just" need to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries and corresponding `.sha256` files to `static.rust-lang.org`.
There is no signature verification in `x.py` as we don't want the build system to depend on GPG. Also, since the checksums were not pinned inside the repository, they were downloaded from `static.rust-lang.org` too: this only protected from accidental changes in `static.rust-lang.org` that didn't change the `*.sha256` files. The attack would allow the attacker to compromise past and future invocations of `x.py`.
## Mitigations introduced in this PR
This PR adds pinned checksums for all the bootstrap components in `src/stage0.json` instead of downloading the checksums from `static.rust-lang.org`. This changes the attack scenario to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries to `static.rust-lang.org`.
3. Land a (reviewed) change in the `rust-lang/rust` repository changing the pinned hashes.
Even with a successful attack, existing clones of the Rust repository won't be affected, and once the attack is detected reverting the pinned hashes changes should be enough to be protected from the attack. This also enables further mitigations to be implemented in following PRs, such as verifying signatures when pinning new checksums (removing the trust on first use aspect of this PR) and adding a check in CI making sure a PR updating the checksum has not been tampered with (see the future improvements section).
## Additional changes
There are additional changes implemented in this PR to enable the mitigation:
* The `src/stage0.txt` file has been replaced with `src/stage0.json`. The reasoning for the change is that there is existing tooling to read and manipulate JSON files compared to the custom format we were using before, and the slight challenge of manually editing JSON files (no comments, no trailing commas) are not a problem thanks to the new `bump-stage0`.
* A new tool has been added to the repository, `bump-stage0`. When invoked, the tool automatically calculates which release should be used as the bootstrap compiler given the current version and channel, gathers all the relevant checksums and updates `src/stage0.json`. The tool can be invoked by running:
```
./x.py run src/tools/bump-stage0
```
* Support for downloading releases from `https://dev-static.rust-lang.org` has been removed, as it's not possible to verify checksums there (it's customary to replace existing artifacts there if a rebuild is warranted). This will require a change to the release process to avoid bumping the bootstrap compiler on beta before the stable release.
## Future improvements
* Add signature verification as part of `bump-stage0`, which would require the attacker to also obtain the release signing keys in order to successfully compromise the bootstrap compiler. This would be fine to add now, as the burden of installing the tool to verify signatures would only be placed on whoever updates the bootstrap compiler, instead of everyone compiling Rust.
* Add a check on CI that ensures the checksums in `src/stage0.json` are the expected ones. If a PR changes the stage0 file CI should also run the `bump-stage0` tool and fail if the output in CI doesn't match the committed file. This prevents the PR author from tweaking the output of the tool manually, which would otherwise be close to impossible for a human to detect.
* Automate creating the PRs bumping the bootstrap compiler, by setting up a scheduled job in GitHub Actions that runs the tool and opens a PR.
* Investigate whether a similar mitigation can be done for "download from CI" components like the prebuilt LLVM.
Auto merge of #88678 - matthewjasper:if-boolean-scoping, r=oli-obk
Change scope of temporaries in match guards
Each pattern in a match arm has its own copy of the match guard in MIR, with its own temporary, so it has to be dropped before the the guards are joined to the single copy of the arm. This PR changes `then_else_break` to allow it to put the temporary in the innermost scope possible. This change isn't done for `if` expressions because that affects a large number of mir-opt tests and could more significantly affect performance.
there is also a random test in here about array repeat expressions that I already had on this branch but it seems to fit the theme of this PR so kept it...
Auto merge of #88631 - camelid:sugg-span, r=davidtwco
Improve structured tuple struct suggestion
Previously, the span was just for the constructor name, which meant it
would result in syntactically-invalid code when applied. Now, the span
is for the entire expression.
I also changed it to use `span_suggestion_verbose`, for two reasons:
1. Now that the suggestion span has been corrected, the output is a bit
cluttered and hard to read. Putting the suggestion its own window
creates more space.
2. It's easier to see what's being suggested, since now the version
after the suggestion is applied is shown.
Auto merge of #88435 - cjgillot:no-walk-crate, r=Aaron1011
Avoid invoking the hir_crate query to traverse the HIR
Walking the HIR tree is done using the `hir_crate` query. However, this is unnecessary, since `hir_owner(CRATE_DEF_ID)` provides the same information. Since depending on `hir_crate` forces dependents to always be executed, this leads to unnecessary work.
By splitting HIR and attributes visits, we can avoid an edge to `hir_crate` when trying to visit the HIR tree.
Stop allocating vtable entries for non-object-safe methods
Current a vtable entry is allocated for all associated fns, even if the method is not object-safe: https://godbolt.org/z/h7vx6f35T
As a result, each vtable for `Iterator`' currently consumes 74 `usize`s. This PR stops allocating vtable entries for those methods, reducing vtable size of each `Iterator` vtable to 7 `usize`s.
Note that this PR introduces will cause more invocations of `is_vtable_safe_method`. So a perf run might be needed. If result isn't favorable then we might need to query-ify `is_vtable_safe_method`.
Matthew Jasper [Sun, 5 Sep 2021 17:50:55 +0000 (18:50 +0100)]
Change scope of temporaries in match guards
Each pattern in a match arm has its own copy of the match guard in MIR,
with its own temporary, so it has to be dropped before the the guards
are joined to the single copy of the arm.
Auto merge of #7638 - xFrednet:7569-avoid-indexing-in-clippy, r=Manishearth
Avoid slice indexing in Clippy (down with the ICEs)
While working on #7569 I got about 23 lint reports where we can avoid slice indexing by destructing the slice early. This is a preparation PR to avoid fixing them in the lint PR. (The implementation already takes about 300 lines without tests :sweat_smile:). Either way, this should hopefully be easy to review :upside_down_face:
Provide `layout_of` automatically (given tcx + param_env + error handling).
After #88337, there's no longer any uses of `LayoutOf` within `rustc_target` itself, so I realized I could move the trait to `rustc_middle::ty::layout` and redesign it a bit.
This is similar to #88338 (and supersedes it), but at no ergonomic loss, since there's no funky `C: LayoutOf<Ty = Ty>` -> `Ty: TyAbiInterface<C>` generic `impl` chain, and each `LayoutOf` still corresponds to one `impl` (of `LayoutOfHelpers`) for the specific context.
After this PR, this is what's needed to get `trait LayoutOf` (with the `layout_of` method) implemented on some context type:
* `TyCtxt`, via `HasTyCtxt`
* `ParamEnv`, via `HasParamEnv`
* a way to transform `LayoutError`s into the desired error type
* an error type of `!` can be paired with having `cx.layout_of(...)` return `TyAndLayout` *without* `Result<...>` around it, such as used by codegen
* this is done through a new `LayoutOfHelpers` trait (and so is specifying the type of `cx.layout_of(...)`)
When going through this path (and not bypassing it with a manual `impl` of `LayoutOf`), the end result is that only the error case can be customized, the query itself and the success paths are guaranteed to be uniform.
(**EDIT**: just noticed that because of the supertrait relationship, you cannot actually implement `LayoutOf` yourself, the blanket `impl` fully covers all possible context types that could ever implement it)
Part of the motivation for this shape of API is that I've been working on querifying `FnAbi::of_*`, and what I want/need to introduce for that looks a lot like the setup in this PR - in particular, it's harder to express the `FnAbi` methods in `rustc_target`, since they're much more tied to `rustc` concepts.
Provide `layout_of` automatically (given tcx + param_env + error handling).
After #88337, there's no longer any uses of `LayoutOf` within `rustc_target` itself, so I realized I could move the trait to `rustc_middle::ty::layout` and redesign it a bit.
This is similar to #88338 (and supersedes it), but at no ergonomic loss, since there's no funky `C: LayoutOf<Ty = Ty>` -> `Ty: TyAbiInterface<C>` generic `impl` chain, and each `LayoutOf` still corresponds to one `impl` (of `LayoutOfHelpers`) for the specific context.
After this PR, this is what's needed to get `trait LayoutOf` (with the `layout_of` method) implemented on some context type:
* `TyCtxt`, via `HasTyCtxt`
* `ParamEnv`, via `HasParamEnv`
* a way to transform `LayoutError`s into the desired error type
* an error type of `!` can be paired with having `cx.layout_of(...)` return `TyAndLayout` *without* `Result<...>` around it, such as used by codegen
* this is done through a new `LayoutOfHelpers` trait (and so is specifying the type of `cx.layout_of(...)`)
When going through this path (and not bypassing it with a manual `impl` of `LayoutOf`), the end result is that only the error case can be customized, the query itself and the success paths are guaranteed to be uniform.
(**EDIT**: just noticed that because of the supertrait relationship, you cannot actually implement `LayoutOf` yourself, the blanket `impl` fully covers all possible context types that could ever implement it)
Part of the motivation for this shape of API is that I've been working on querifying `FnAbi::of_*`, and what I want/need to introduce for that looks a lot like the setup in this PR - in particular, it's harder to express the `FnAbi` methods in `rustc_target`, since they're much more tied to `rustc` concepts.
Change return type for T::{log,log2,log10} to u32. The value is at
most 128, and this is consistent with using u32 for small values
elsewhere (e.g. BITS, count_ones, leading_zeros).
Auto merge of #7627 - xFrednet:0000-updating-issue-templates-again, r=llogiq
Updating issue templates again for rustbot
It turns out that our current issue template can sometimes trigger a rustbot error message, as can be seen in [#7626](https://github.com/rust-lang/rust-clippy/issues/7626). I originally tested this in #7599, but it's apparently a bit inconsistent. This PR adds backticks to the commands, as correctly suggested by `@mikerite` in the comments. (Thank you!)
``@rustbot` label +S-blocked`
---
Now I also pushed a tiny link fix as well. :upside_down_face:
Auto merge of #88604 - camelid:rustdoc-lifetime-bounds, r=GuillaumeGomez
rustdoc: Clean up handling of lifetime bounds
Previously, rustdoc recorded lifetime bounds by rendering them into the
name of the lifetime parameter. Now, it leaves the name as the actual
name and instead records lifetime bounds in an `outlives` list, similar
to how type parameter bounds are recorded.
Also, higher-ranked lifetimes cannot currently have bounds, so I simplified
the code to reflect that.
Auto merge of #88662 - m-ou-se:rollup-h6lgp7k, r=m-ou-se
Rollup of 4 pull requests
Successful merges:
- #88257 (Provide more context on incorrect inner attribute)
- #88432 (Fix a typo in raw_vec)
- #88511 (x.py clippy: don't run with --all-targets by default)
- #88657 (Fix 2021 `dyn` suggestion that used code as label)
Auto merge of #88469 - patrick-gu:master, r=dtolnay
Add links in docs for some primitive types
This pull request adds additional links in existing documentation of some of the primitive types.
Where items are linked only once, I have used the `[link](destination)` format. For items in `std`, I have linked directly to the HTML, since although the primitives are in `core`, they are not displayed on `core` documentation. I was unsure of what length I should keep lines of documentation to, so I tried to keep them within reason.
Additionally, I have avoided excessively linking to keywords like `self` when they are not relevant to the documentation. I can add these links if it would be an improvement.
I hope this can improve Rust. Please let me know if there's anything I did wrong!
Auto merge of #88364 - pietroalbini:llvm-install-filecheck, r=Mark-Simulacrum
Make sure FileCheck is copied in the LLVM output directory
The tool, which is needed by parts of our test suite, is built as part of LLVM but is *not* copied to the directory containing the output LLVM binaries. This adds a flag to ensure the binary is copied. This shouldn't add any extra built time, as the flag just installs extra binaries that were already compiled.
This is not strictly needed for the test suite to work (as it also checks `build/$target/llvm/build/bin` for the binary), but it allows deleting the `build/$TARGET/llvm/build` directory (which also contains the intermediary build artifacts) without affecting the test suite, saving disk space.
Kevin Reid [Sat, 4 Sep 2021 18:05:36 +0000 (11:05 -0700)]
“Moves” instead of “copies” in `<Option<T> as From<T>>::from` doc.
This implementation has no `Copy` or `Clone` bound, so its operation is
guaranteed to be a move. The call site might copy, but the function
itself cannot.
Also linkify `Some` while we're touching the line anyway.
Auto merge of #88550 - dpaoliello:dpaoliello/allocdebuginfo, r=estebank
Include debug info for the allocator shim
Issue Details:
In some cases it is necessary to generate an "allocator shim" to forward various Rust allocation functions (e.g., `__rust_alloc`) to an underlying function (e.g., `malloc`). However, since this allocator shim is a manually created LLVM module it is not processed via the normal module processing code and so no debug info is generated for it (if debugging info is enabled).
Fix Details:
* Modify the `debuginfo` code to allow creating debug info for a module without a `CodegenCx` (since it is difficult, and expensive, to create one just to emit some debug info).
* After creating the allocator shim add in basic debug info.
Auto merge of #88538 - bjorn3:no_session_in_crate_loader, r=petrochenkov
CrateLocator refactorings
This makes the `CrateLocator` a lot cleaner IMHO and much more self-contained. The last commit removes `extra_filename` from the crate metadata. This is an **insta-stable** change as it allows a crate like `libfoo-abc.rlib` to be used as dependency and then be renamed as `libfoo-bcd.rlib` while still being found as indirect dependency. This may reduce performance when there are a lot of versions of the same crate available as the extra filename won't be used to do an early rejection of crates before trying to load metadata, but it makes the logic to find the right filename a lot cleaner.
Auto merge of #88482 - athei:add-import-test, r=the8472
Add regression test for a spurious import
This PR adds a test that verifies that the bug described in the linked issue does not creep back into the code. In essence it checks that compiling some specific code (that uses 128 bit multiplication) with a specific set of compiler options does not lead to a spurious import of a panic function.
I noticed that other wasm tests use `# only-wasm32-bare` in their `Makefile`. This will skip the test for me. I did not find out how to run this test locally. Maybe someone can help.
Auto merge of #88572 - matthewjasper:if-let-scoping-fix, r=oli-obk
Fix drop handling for `if let` expressions
MIR lowering for `if let` expressions is now more complicated now that
`if let` exists in HIR. This PR adds a scope for the variables bound in
an `if let` expression and then uses an approach similar to how we
handle loops to ensure that we reliably drop the correct variables.
Closes #88307
cc `@flip1995` `@richkadel` `@c410-f3r`
Auto merge of #88618 - m-ou-se:rollup-6tss5z6, r=m-ou-se
Rollup of 7 pull requests
Successful merges:
- #88202 (Add an example for deriving PartialOrd on enums)
- #88483 (Fix LLVM libunwind build for non-musl targets)
- #88507 (Add test case for using `slice::fill` with MaybeUninit)
- #88557 (small const generics cleanup)
- #88579 (remove redundant / misplaced sentence from docs)
- #88610 (Update outdated docs of array::IntoIter::new.)
- #88613 (Update primitive docs for rust 2021.)
Mara Bos [Fri, 3 Sep 2021 11:30:49 +0000 (13:30 +0200)]
Rollup merge of #88579 - ast-ral:master, r=m-ou-se
remove redundant / misplaced sentence from docs
Removes sentence that seems to have drifted down into the examples section. Luckily, someone already added an explanation of what happens with packed structs back into the initial section of the doc entry and this wayward sentence can likely just be deleted.
projects / rust.git / log