bors [Mon, 15 Mar 2021 08:49:25 +0000 (08:49 +0000)]
Auto merge of #82999 - cuviper:rustc-rayon-0.3.1, r=Mark-Simulacrum
Update to rustc-rayon 0.3.1
This pulls in rust-lang/rustc-rayon#8 to fix #81425. (h/t `@ammaraskar)`
That revealed weak constraints on `rustc_arena::DropArena`, because its
`DropType` was holding type-erased raw pointers to generic `T`. We can
implement `Send` for `DropType` (under `cfg(parallel_compiler)`) by
requiring all `T: Send` before they're type-erased.
bors [Mon, 15 Mar 2021 06:20:24 +0000 (06:20 +0000)]
Auto merge of #83074 - Aaron1011:new-sort-fix, r=jackh726
Avoid sorting predicates by `DefId`
Fixes issue #82920
Even if an item does not change between compilation sessions, it may end
up with a different `DefId`, since inserting/deleting an item affects
the `DefId`s of all subsequent items. Therefore, we use a `DefPathHash`
in the incremental compilation system, which is stable in the face of
changes to unrelated items.
In particular, the query system will consider the inputs to a query to
be unchanged if any `DefId`s in the inputs have their `DefPathHash`es
unchanged. Queries are pure functions, so the query result should be
unchanged if the query inputs are unchanged.
Unfortunately, it's possible to inadvertantly make a query result
incorrectly change across compilations, by relying on the specific value
of a `DefId`. Specifically, if the query result is a slice that gets
sorted by `DefId`, the precise order will depend on how the `DefId`s got
assigned in a particular compilation session. If some definitions end up
with different `DefId`s (but the same `DefPathHash`es) in a subsequent
compilation session, we will end up re-computing a *different* value for
the query, even though the query system expects the result to unchanged
due to the unchanged inputs.
It turns out that we have been sorting the predicates computed during
`astconv` by their `DefId`. These predicates make their way into the
`super_predicates_that_define_assoc_type`, which ends up getting used to
compute the vtables of trait objects. This, re-ordering these predicates
between compilation sessions can lead to undefined behavior at runtime -
the query system will re-use code built with a *differently ordered*
vtable, resulting in the wrong method being invoked at runtime.
This PR avoids sorting by `DefId` in `astconv`, fixing the
miscompilation. However, it's possible that other instances of this
issue exist - they could also be easily introduced in the future.
To fully fix this issue, we should
1. Turn on `-Z incremental-verify-ich` by default. This will cause the
compiler to ICE whenver an 'unchanged' query result changes between
compilation sessions, instead of causing a miscompilation.
2. Remove the `Ord` impls for `CrateNum` and `DefId`. This will make it
difficult to introduce ICEs in the first place.
bors [Sun, 14 Mar 2021 04:09:02 +0000 (04:09 +0000)]
Auto merge of #83105 - JohnTitor:rollup-tqpm8pb, r=JohnTitor
Rollup of 10 pull requests
Successful merges:
- #81465 (Add documentation about formatting `Duration` values)
- #82121 (Implement Extend and FromIterator for OsString)
- #82617 (Document `everybody_loops`)
- #82789 (Get with field index from pattern slice instead of directly indexing)
- #82798 (Rename `rustdoc` to `rustdoc::all`)
- #82804 (std: Fix a bug on the wasm32-wasi target opening files)
- #82943 (Demonstrate best practice for feeding stdin of a child processes)
- #83066 (Add `reverse` search alias for Iterator::rev())
- #83070 (Update cargo)
- #83081 (Fix panic message of `assert_failed_inner`)
Yuki Okushi [Sun, 14 Mar 2021 04:07:36 +0000 (13:07 +0900)]
Rollup merge of #83070 - ehuss:update-cargo, r=ehuss
Update cargo
7 commits in 970bc67c3775781b9708c8a36893576b9459c64a..32da9eaa5de5be241cf8096ca6b749a157194f77
2021-03-07 18:09:40 +0000 to 2021-03-13 01:18:40 +0000
- Fix logic for determining prefer-dynamic for a dylib. (rust-lang/cargo#9252)
- Fix issue with filtering exclusive target dependencies. (rust-lang/cargo#9255)
- Update pkgid-spec docs. (rust-lang/cargo#9249)
- Wordsmith the edition documentation a bit more (rust-lang/cargo#9233)
- Package ID specification urls must contain a host (rust-lang/cargo#9188)
- Add documentation for JSON message_path. (rust-lang/cargo#9247)
- Fix filter_platform to run on targets other than x86. (rust-lang/cargo#9246)
Yuki Okushi [Sun, 14 Mar 2021 04:07:34 +0000 (13:07 +0900)]
Rollup merge of #82943 - kornelski:threadstdio, r=joshtriplett
Demonstrate best practice for feeding stdin of a child processes
Documentation change.
It's possible to create a deadlock with stdin/stdout I/O on a single thread:
* the child process may fill its stdout buffer, and have to wait for the parent process to read it,
* but the parent process may be waiting until its stdin write finishes before reading the stdout.
Therefore, the parent process should use separate threads for writing and reading.
These examples are not deadlocking in practice, because they use short strings, but I think it's better to demonstrate code that works even for long writes. The problem is non-obvious and tricky to debug (it seems that even libstd has a similar issue: #45572).
This also demonstrates how to use stdio with threads: it's not obvious that `.take()` can be used to avoid fighting with the borrow checker.
Yuki Okushi [Sun, 14 Mar 2021 04:07:33 +0000 (13:07 +0900)]
Rollup merge of #82804 - alexcrichton:fix-wasi, r=pnkfelix
std: Fix a bug on the wasm32-wasi target opening files
This commit fixes an issue pointed out in #82758 where LTO changed the
behavior of a program. It turns out that LTO was not at fault here, it
simply uncovered an existing bug. The bindings to
`__wasilibc_find_relpath` assumed that the relative portion of the path
returned was always contained within thee input `buf` we passed in. This
isn't actually the case, however, and sometimes the relative portion of
the path may reference a sub-portion of the input string itself.
The fix here is to use the relative path pointer coming out of
`__wasilibc_find_relpath` as the source of truth. The `buf` used for
local storage is discarded in this function and the relative path is
copied out unconditionally. We might be able to get away with some
`Cow`-like business or such to avoid the extra allocation, but for now
this is probably the easiest patch to fix the original issue.
Yuki Okushi [Sun, 14 Mar 2021 04:07:31 +0000 (13:07 +0900)]
Rollup merge of #82789 - csmoe:issue-82772, r=estebank
Get with field index from pattern slice instead of directly indexing
Closes #82772
r? ``@estebank``
https://github.com/rust-lang/rust/pull/82789#issuecomment-796921977
> ``@estebank`` So the real cause is we only generate single pattern for Box here
https://github.com/csmoe/rust/blob/615b03aeaa8ce9819de7828740ab3cd7def4fa76/compiler/rustc_mir_build/src/thir/pattern/deconstruct_pat.rs#L1130-L1132
But in the replacing function, it tries to index on the 1-length pattern slice with field 1, thus out of bounds.
https://github.com/csmoe/rust/blob/615b03aeaa8ce9819de7828740ab3cd7def4fa76/compiler/rustc_mir_build/src/thir/pattern/deconstruct_pat.rs#L1346
I use `clap` for command line argument parsing, which collects these `-e` commands into a `Vec<OsString>`. To pass these commands to the interpreter for `Eval`, I need to join them together. Combining these impls with `Iterator::intersperse` https://github.com/rust-lang/rust/issues/79524 would enable me to build a single bit of Ruby code.
Currently, I'm doing something like:
```rust
let mut commands = commands.into_iter();
let mut buf = if let Some(command) = commands.next() {
command
} else {
return Ok(Ok(()));
};
for command in commands {
buf.push("\n");
buf.push(command);
}
```
If there's interest, I'd also like to add impls for `Cow<'a, OsStr>`, which would avoid allocating the `"\n"` `OsString` in the concatenate + intersperse use case.
Aaron Hill [Fri, 12 Mar 2021 22:53:02 +0000 (17:53 -0500)]
Avoid sorting predicates by `DefId`
Fixes issue #82920
Even if an item does not change between compilation sessions, it may end
up with a different `DefId`, since inserting/deleting an item affects
the `DefId`s of all subsequent items. Therefore, we use a `DefPathHash`
in the incremental compilation system, which is stable in the face of
changes to unrelated items.
In particular, the query system will consider the inputs to a query to
be unchanged if any `DefId`s in the inputs have their `DefPathHash`es
unchanged. Queries are pure functions, so the query result should be
unchanged if the query inputs are unchanged.
Unfortunately, it's possible to inadvertantly make a query result
incorrectly change across compilations, by relying on the specific value
of a `DefId`. Specifically, if the query result is a slice that gets
sorted by `DefId`, the precise order will depend on how the `DefId`s got
assigned in a particular compilation session. If some definitions end up
with different `DefId`s (but the same `DefPathHash`es) in a subsequent
compilation session, we will end up re-computing a *different* value for
the query, even though the query system expects the result to unchanged
due to the unchanged inputs.
It turns out that we have been sorting the predicates computed during
`astconv` by their `DefId`. These predicates make their way into the
`super_predicates_that_define_assoc_type`, which ends up getting used to
compute the vtables of trait objects. This, re-ordering these predicates
between compilation sessions can lead to undefined behavior at runtime -
the query system will re-use code built with a *differently ordered*
vtable, resulting in the wrong method being invoked at runtime.
This PR avoids sorting by `DefId` in `astconv`, fixing the
miscompilation. However, it's possible that other instances of this
issue exist - they could also be easily introduced in the future.
To fully fix this issue, we should
1. Turn on `-Z incremental-verify-ich` by default. This will cause the
compiler to ICE whenver an 'unchanged' query result changes between
compilation sessions, instead of causing a miscompilation.
2. Remove the `Ord` impls for `CrateNum` and `DefId`. This will make it
difficult to introduce ICEs in the first place.
Aaron Hill [Thu, 11 Mar 2021 06:12:07 +0000 (01:12 -0500)]
Always run `incremental_verify_ich` when re-computing query results
Issue #82920 showed that the kind of bugs caught by this flag have
soundness implications.
This causes performance regressions of up to 15.2% during incremental
compilation, but this is necessary to catch miscompilations caused by
bugs in query implementations.
bors [Sat, 13 Mar 2021 09:47:07 +0000 (09:47 +0000)]
Auto merge of #82878 - sexxi-goose:repr_packed, r=nikomatsakis
2229: Handle capturing a reference into a repr packed struct
RFC 1240 states that it is unsafe to capture references into a
packed-struct. This PR ensures that when a closure captures a precise
path, we aren't violating this safety constraint.
To acheive so we restrict the capture precision to the struct itself.
An interesting edge case where we decided to restrict precision:
```rust
struct Foo(String);
let foo: Foo;
let c = || {
println!("{}", foo.0);
let x = foo.0;
}
```
Given how closures get desugared today, foo.0 will be moved into the
closure, making the `println!`, safe. However this can be very subtle
and also will be unsafe if the closure gets inline.
bors [Sat, 13 Mar 2021 04:38:39 +0000 (04:38 +0000)]
Auto merge of #82436 - osa1:issue80258, r=nikomatsakis
Allow calling *const methods on *mut values
This allows `*const` methods to be called on `*mut` values.
TODOs:
- [x] ~~Remove debug logs~~ Done.
- [x] ~~I haven't tested, but I think this currently won't work when the `self` value has type like `&&&&& *mut X` because I don't do any autoderefs when probing. To fix this the new code in `rustc_typeck::check::method::probe` needs to reuse `pick_method` somehow as I think that's the function that autoderefs.~~ This works, because autoderefs are done before calling `pick_core`, in `method_autoderef_steps`, called by `probe_op`.
- [x] ~~I should probably move the new `Pick` to `pick_autorefd_method`. If not, I should move it to its own function.~~ Done.
- [ ] ~~Test this with a `Pick` with `to_ptr = true` and `unsize = true`.~~ I think this case cannot happen, because we don't have any array methods with `*mut [X]` receiver. I should confirm that this is true and document this. I've placed two assertions about this.
- [x] ~~Maybe give `(Mutability, bool)` a name and fields~~ I now have a `to_const_ptr` field in `Pick`.
- [x] ~~Changes in `adjust_self_ty` is quite hacky. The problem is we can't deref a pointer, and even if we don't have an adjustment to get the address of a value, so to go from `*mut` to `*const` we need a special case.~~ There's still a special case for `to_const_ptr`, but I'm not sure if we can avoid this.
- [ ] Figure out how `reached_raw_pointer` stuff is used. I suspect only for error messages.
bors [Sat, 13 Mar 2021 02:08:33 +0000 (02:08 +0000)]
Auto merge of #83067 - JohnTitor:rollup-0wo338i, r=JohnTitor
Rollup of 6 pull requests
Successful merges:
- #82984 (Simplify ast block lowering)
- #83012 (Update Clippy)
- #83020 (Emit the enum range assumption if the range only contains one element)
- #83037 (Support merge_functions option in NewPM since LLVM >= 12)
- #83052 (updated vulnerable deps)
- #83059 (Allow configuring `rustdoc --disable-minification` in config.toml)
Yuki Okushi [Sat, 13 Mar 2021 00:44:46 +0000 (09:44 +0900)]
Rollup merge of #83052 - klensy:dep-update, r=Mark-Simulacrum
updated vulnerable deps
* Updated signal-hook-registry 1.2.1 to 1.2.2, as it drops dependency on vulnerable arc-swap 0.4.7 https://rustsec.org/advisories/RUSTSEC-2020-0091
* Updated generic-array 0.12.3 to 0.12.4, vuln https://rustsec.org/advisories/RUSTSEC-2020-0146
* Updated sized-chunks 0.6.2 to 0.6.4, vuln https://rustsec.org/advisories/RUSTSEC-2020-0041 fixed in 0.6.3, in 0.6.4 fixed some UB https://github.com/bodil/sized-chunks/blob/master/CHANGELOG.md#064---2021-02-17
bors [Fri, 12 Mar 2021 23:27:23 +0000 (23:27 +0000)]
Auto merge of #83022 - m-ou-se:mem-replace-no-swap, r=nagisa
Don't implement mem::replace with mem::swap.
`swap` is a complicated operation, so this changes the implementation of `replace` to use `read` and `write` instead.
See https://github.com/rust-lang/rust/pull/83019.
I wrote there:
> Implementing the simpler operation (replace) with the much more complicated operation (swap) doesn't make a whole lot of sense. `replace` is just read+write, and the primitive for moving out of a `&mut`. `swap` is for doing that to *two* `&mut` at the same time, which is both more niche and more complicated (as shown by `swap_nonoverlapping_bytes`).
This could be especially interesting for `Option<VeryLargeStruct>::take()`, since swapping such a large structure with `swap_nonoverlapping_bytes` is going to be much less efficient than `ptr::write()`'ing a `None`.
But also for small values where `swap` just reads/writes using temporary variable, this makes a `replace` or `take` operation simpler:
![image](https://user-images.githubusercontent.com/783247/110839393-c7e6bd80-82a3-11eb-97b7-28acb14deffd.png)
bors [Fri, 12 Mar 2021 11:46:50 +0000 (11:46 +0000)]
Auto merge of #82422 - petrochenkov:allunst, r=oli-obk
expand: Do not allocate `Lrc` for `allow_internal_unstable` list unless necessary
This allocation is done for any macro defined in the current crate, or used from a different crate.
EDIT: This also removes an `Lrc` increment from each *use* of such macro, which may be more significant.
Noticed when reviewing https://github.com/rust-lang/rust/pull/82367.
This probably doesn't matter, but let's do a perf run.
bors [Fri, 12 Mar 2021 00:23:05 +0000 (00:23 +0000)]
Auto merge of #83042 - JohnTitor:rollup-s8efv94, r=JohnTitor
Rollup of 11 pull requests
Successful merges:
- #80385 (Clarify what `Cell::replace` returns)
- #82571 (Rustdoc Json: Add tests for Reexports, and improve jsondocck)
- #82860 (Add `-Z unpretty` flag for the THIR)
- #82950 (convert slice doc link to intra-doc links)
- #82965 (Add spirv extension handling in compiletest)
- #82966 (update MSYS2 link in README)
- #82979 (Fix "run" button position in error index)
- #83001 (Ignore Vim swap files)
- #83003 (rustdoc: tweak the search index format)
- #83013 (Adjust some `#[cfg]`s to take non-Unix non-Windows operating systems into account)
- #83018 (Reintroduce accidentally deleted assertions.)
That's a 4% improvement on the uncompressed version (fewer `[]`, and also changing `null` to `0` in the parent_idx list), and 20% improvement after gzipping it, thanks to putting like-typed data next to each other. Any compression algorithm based on a sliding window will probably show this kind of improvement.
Yuki Okushi [Thu, 11 Mar 2021 23:55:15 +0000 (08:55 +0900)]
Rollup merge of #82950 - mockersf:slice-intra-doc-link, r=jyn514
convert slice doc link to intra-doc links
Continuing where #80189 stopped, with `core::slice`.
I had an issue with two dead links in my doc when implementing `Deref<Target = [T]>` for one of my type. This means that [`binary_search_by_key`](https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.binary_search_by_key) was available, but not [`sort_by_key`](https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.sort_by_key) even though it was linked in it's doc (same issue with [`as_ptr`](https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_ptr) and [`as_mut_pbr`](https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_mut_ptr)). It becomes available if I implement `DerefMut`, as it needs an `&mut self`.
<details>
<summary>Code that will have dead links in its doc</summary>
I removed the link to `sort_by_key` from `binary_search_by_key` doc as I didn't find a nice way to have a live link:
- `binary_search_by_key` is in `core`
- `sort_by_key` is in `alloc`
- intra-doc link `slice::sort_by_key` doesn't work, as `alloc` is not available when `core` is being build (the warning can't be ignored: ```error[E0710]: an unknown tool name found in scoped lint: `rustdoc::broken_intra_doc_links` ```)
- keeping the link as an anchor `#method.sort_by_key` meant a dead link
- an absolute link would work but doesn't feel right...
Yuki Okushi [Thu, 11 Mar 2021 23:55:14 +0000 (08:55 +0900)]
Rollup merge of #82860 - LeSeulArtichaut:unpretty-thir, r=spastorino
Add `-Z unpretty` flag for the THIR
This adds a new perma-unstable flag, `-Zunpretty=thir-tree`, that dumps the raw THIR tree for each body in the crate.
Implements the THIR part of MCP rust-lang/compiler-team#408, helps with rust-lang/rustc-dev-guide#1062.
Depends on #82495, blocked on that. Only the two last commits are added by this PR.
r? ```@spastorino``` cc ```@estebank```
Yuki Okushi [Thu, 11 Mar 2021 23:55:13 +0000 (08:55 +0900)]
Rollup merge of #82571 - aDotInTheVoid:reexport-tests, r=CraftSpider
Rustdoc Json: Add tests for Reexports, and improve jsondocck
The two changes are orthognal, so you can land just one if you want, but the improved errors realy helped write the tests.
Notably does not have the case from #80664, but I want to have all the ajacent cases tested before starting work on that to ensure I dont break anything.
bors [Thu, 11 Mar 2021 21:41:01 +0000 (21:41 +0000)]
Auto merge of #82417 - the8472:fix-copy_file_range-append, r=m-ou-se
Fix io::copy specialization using copy_file_range when writer was opened with O_APPEND
fixes #82410
While `sendfile()` returns `EINVAL` when the output was opened with O_APPEND, `copy_file_range()` does not and returns `EBADF` instead, which – unlike other `EBADF` causes – is not fatal for this operation since a regular `write()` will likely succeed.
We now treat `EBADF` as a non-fatal error for `copy_file_range` and fall back to a read-write copy as we already did for several other errors.
bors [Thu, 11 Mar 2021 18:14:59 +0000 (18:14 +0000)]
Auto merge of #82806 - nikic:memcpyopt-mssa, r=nagisa
Enable MemorySSA in MemCpyOpt
LLVM 12 ships with an implementation of MemCpyOpt which is based on MSSA instead of MDA. This implementation can eliminate memcpys across blocks, and as such fixes many (but not all) failures to eliminate redundant memcpys for Rust code. Unfortunately this was only enabled by default shortly after LLVM 12 was cut. This backports the enablement to our LLVM fork.
There are improvements on check and debug builds, which indicate that rustc itself has become faster. For opt builds this is, on average, a very minor improvement as well, although there is one significant outlier with deep-vector-opt. This benchmark creates ~140000 zero stores, which are now coalesced into a memset slightly later, resulting in longer compile-time for intermediate passes.
bors [Thu, 11 Mar 2021 16:10:33 +0000 (16:10 +0000)]
Auto merge of #6884 - matthiaskrgr:lintcheck_crate, r=Manishearth
move lintcheck into its own crate
This pr:
* moves lintcheck out of `clippy dev` and into its own crate (`lintcheck`) (I should have done this earlier :D)
* makes lintcheck terminate if it is not launched from the repo root (to prevent problems with wrong paths when using `cargo run` in the crate root)
* fixes json lint messages leaking the runners `$HOME` when a lint messages comes from a proc macro that originates from a crate inside the `$CARGO_CACHE`
* adds more documentation to lintchecks `README.md` and mentions lintcheck in `docs/basics.md`
bors [Thu, 11 Mar 2021 15:34:01 +0000 (15:34 +0000)]
Auto merge of #82495 - LeSeulArtichaut:eager-thir, r=oli-obk
Eagerly construct bodies of THIR
With this PR:
- the THIR is no longer constructed lazily, but is entirely built before being passed to the MIR Builder
- the THIR is now allocated in arenas instead of `Box`es
However, this PR doesn't make any changes to the way patterns are constructed: they are still boxed, and exhaustiveness checking is unchanged.
Implements MCP rust-lang/compiler-team#409.
Closes rust-lang/project-thir-unsafeck#1.
r? `@ghost` cc `@nikomatsakis` `@oli-obk`