Dylan DPC [Sun, 15 Nov 2020 02:02:51 +0000 (03:02 +0100)]
Rollup merge of #78969 - tmiasko:normalize, r=davidtwco
Normalize function type during validation
During inlining, the callee body is normalized and has types revealed,
but some of locals corresponding to the arguments might come from the
caller body which is not. As a result the caller body does not pass
validation without additional normalization.
Dylan DPC [Sun, 15 Nov 2020 02:02:49 +0000 (03:02 +0100)]
Rollup merge of #78968 - zec:add-llvm-as, r=Mark-Simulacrum
Include llvm-as in llvm-tools-preview component
Including `llvm-as` adds the ability to include assembly language fragments that can be inlined using LTO while making sure the correct version of LLVM is always used.
This is pretty much self contained, but depending on feedback and timing, I may have a chance to add a few more unit tests requested against `counters.rs`. I'm looking at those now.
Dylan DPC [Sun, 15 Nov 2020 02:02:39 +0000 (03:02 +0100)]
Rollup merge of #78848 - DevJPM:ci-llvm-9, r=nikic
Bump minimal supported LLVM version to 9
This bumps the minimal tested llvm version to 9.
This should enable supporting newer LLVM features (and CPU extensions).
This was motived by #78361 having to drop features because of LLVM 8 not supporting certain CPU extensions yet.
This was declared relatively uncontroversial on [Zulip](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Min.20Supported.20LLVM.20Upgrade.20Process.3F/near/215957859).
Paging ````@eddyb```` because there was a comment in the [dockerfile](https://github.com/rust-lang/rust/blob/master/src/ci/docker/host-x86_64/x86_64-gnu-llvm-8/Dockerfile#L42) describing a hack (which I don't quite understand) which was also blocked by not having LLVM 9.
This PR removes the need for ignore-tidy-filelength for alloc::collections::vec_deque which is part of the issue https://github.com/rust-lang/rust/issues/60302
It is probably easiest to review this PR by looking at it commit by commit rather than looking at the overall diff.
This allows constant propagation to evaluate `size_of` and `wrapping_*`,
and unreachable propagation to propagate a call to `unreachable`.
The lowering is performed as a MIR optimization, rather than during MIR
building to preserve the special status of intrinsics with respect to
unsafety checks and promotion.
Currently enabled by default to determine the performance impact (no
significant impact expected). In practice only useful when combined with
inlining since intrinsics are rarely used directly (with exception of
`unreachable` and `discriminant_value` used by built-in derive macros).
bors [Sat, 14 Nov 2020 18:03:17 +0000 (18:03 +0000)]
Auto merge of #78809 - vn-ki:fix-issue-76064, r=oli-obk
add error_occured field to ConstQualifs,
fix #76064
I wasn't sure what `in_return_place` actually did and not sure why it returns `ConstQualifs` while it's sibling functions return `bool`. So I tried to make as minimal changes to the structure as possible. Please point out whether I have to refactor it or not.
bors [Sat, 14 Nov 2020 12:01:55 +0000 (12:01 +0000)]
Auto merge of #75272 - the8472:spec-copy, r=KodrAus
specialize io::copy to use copy_file_range, splice or sendfile
Fixes #74426.
Also covers #60689 but only as an optimization instead of an official API.
The specialization only covers std-owned structs so it should avoid the problems with #71091
Currently linux-only but it should be generalizable to other unix systems that have sendfile/sosplice and similar.
There is a bit of optimization potential around the syscall count. Right now it may end up doing more syscalls than the naive copy loop when doing short (<8KiB) copies between file descriptors.
0-1 `stat` calls to identify the source file type. 0 if the type can be inferred from the struct from which the FD was extracted
𝖬 `write` to drain the `BufReader`/`BufWriter` wrappers. only happen when buffers are present. 𝖬 ≾ number of wrappers present. If there is a write buffer it may absorb the read buffer contents first so only result in a single write. Vectored writes would also be an option but that would require more invasive changes to `BufWriter`.
𝖭 `copy_file_range`/`splice`/`sendfile` until file size, EOF or the byte limit from `Take` is reached. This should generally be *much* more efficient than the read-write loop and also have other benefits such as DMA offload or extent sharing.
bors [Sat, 14 Nov 2020 04:21:56 +0000 (04:21 +0000)]
Auto merge of #78736 - petrochenkov:lazyenum, r=Aaron1011
rustc_parse: Remove optimization for 0-length streams in `collect_tokens`
The optimization conflates empty token streams with unknown token stream, which is at least suspicious, and doesn't affect performance because 0-length token streams are very rare.
This allows constant propagation to evaluate `size_of` and `wrapping_*`,
and unreachable propagation to propagate a call to `unreachable`.
The lowering is performed as a MIR optimization, rather than during MIR
building to preserve the special status of intrinsics with respect to
unsafety checks and promotion.
The8472 [Thu, 12 Nov 2020 22:39:49 +0000 (23:39 +0100)]
Always handle EOVERFLOW by falling back to the generic copy loop
Previously EOVERFLOW handling was only applied for io::copy specialization
but not for fs::copy sharing the same code.
Additionally we lower the chunk size to 1GB since we have a user report
that older kernels may return EINVAL when passing 0x8000_0000
but smaller values succeed.
The8472 [Fri, 6 Nov 2020 22:36:23 +0000 (23:36 +0100)]
do direct splice syscall and probe availability to get android builds to work
Android builds use feature level 14, the libc wrapper for splice is gated
on feature level 21+ so we have to invoke the syscall directly.
Additionally the emulator doesn't seem to support it so we also have to
add ENOSYS checks.
bors [Fri, 13 Nov 2020 20:06:46 +0000 (20:06 +0000)]
Auto merge of #78888 - richkadel:llvm-coverage-tests, r=tmandry
Fix and re-enable two coverage tests on MacOS
Note, in the coverage-reports test, the comment about MacOS was wrong.
The setting is based on config.toml llvm `optimize` setting. There
doesn't appear to be any environment variable I can check, and I
don't think we should add one. Testing the binary itself is a more
reliable way to check anyway.
For the coverage-spanview test, I removed the dependency on sed
altogether, which is much less ugly than trying to work around the
MacOS sed differences.
I tested these changes on Linux, Windows, and Mac.
bors [Fri, 13 Nov 2020 14:53:55 +0000 (14:53 +0000)]
Auto merge of #79017 - GuillaumeGomez:rollup-5orhudd, r=GuillaumeGomez
Rollup of 6 pull requests
Successful merges:
- #77151 (Add regression test for issue #76042)
- #77996 (Doc change: Remove mention of `fnv` in HashMap)
- #78463 (Add type to `ConstKind::Placeholder`)
- #78984 (Rustdoc check option)
- #78985 (add dropck test for const params)
- #78996 (add explicit test for const param promotion)
Guillaume Gomez [Fri, 13 Nov 2020 14:26:10 +0000 (15:26 +0100)]
Rollup merge of #77996 - tkaitchuck:master, r=m-ou-se
Doc change: Remove mention of `fnv` in HashMap
Disclaimer: I am the author of [aHash](https://github.com/tkaitchuck/aHash).
This changes the Rustdoc in `HashMap` from mentioning the `fnv` crate to mentioning the `aHash` crate, as an alternative `Hasher` implementation.
### Why
Fnv [has poor hash quality](https://github.com/rurban/smhasher), is [slow for larger keys](https://github.com/tkaitchuck/aHash/blob/master/compare/readme.md#speed), and does not provide dos resistance, because it is unkeyed (this can also cause [other problems](https://accidentallyquadratic.tumblr.com/post/153545455987/rust-hash-iteration-reinsertion)).
Fnv has acceptable performance for integers and has very poor performance with keys >32 bytes. This is the reason it was removed from the standard library in https://github.com/rust-lang/rust/pull/37229 .
Because regardless of which dimension you value, there are better alternatives, it does not make sense for anyone to consider using `fnv`.
The text mentioning `fnv` in the standard library continues to create confusion: https://github.com/rust-lang/hashbrown/issues/153 https://github.com/rust-lang/hashbrown/issues/9 . There are also a number of [crates using it](https://crates.io/crates/fnv/reverse_dependencies) a great many of which are hashing strings (Which is when Fnv is the [worst](https://github.com/cbreeden/fxhash#benchmarks), [possible](https://github.com/tkaitchuck/aHash#speed), [choice](http://cglab.ca/~abeinges/blah/hash-rs/).)
I think aHash makes the most sense to mention as an alternative because it is the most credible option (in my obviously biased opinion). It offers [good performance on numbers and strings](https://github.com/tkaitchuck/aHash/blob/master/compare/readme.md#speed), is [of high quality](https://github.com/tkaitchuck/aHash#hash-quality), and [provides dos resistance](https://github.com/tkaitchuck/aHash/wiki/How-aHash-is-resists-DOS-attacks). It is popular (see [stats](https://crates.io/crates/ahash)) and is the default hasher for [hashbrown](https://crates.io/crates/hashbrown) and [dashmap](https://crates.io/crates/dashmap) which are the most popular alternative hashmaps. Finally it does not have any of the [`gotcha` cases](https://github.com/tkaitchuck/aHash#fxhash) that `FxHash` suffers from. (Which is the other popular hashing option when DOS attacks are not a concern)
Signed-off-by: Tom Kaitchuck <tom.kaitchuck@emc.com>
bors [Fri, 13 Nov 2020 11:49:33 +0000 (11:49 +0000)]
Auto merge of #77990 - bugadani:copies, r=lcnr
Eliminate some temporary vectors
This PR changes `get_item_attrs` and `get_item_variances` to return iterator impls instead of vectors. On top of that, this PR replaces some seemingly unnecessary vectors with iterators or SmallVec, and also reserves space where we know (the minimum) number of elements that will be inserted. This change hopes to remove a few heap allocations and unnecessary copies.
bors [Fri, 13 Nov 2020 05:40:37 +0000 (05:40 +0000)]
Auto merge of #78826 - petrochenkov:mrscopes2, r=eddyb
resolve: Collapse `macro_rules` scope chains on the fly
Otherwise they grow too long and you have to endlessly walk through them when resolving macros or imports.
Addresses https://rust-lang.zulipchat.com/#narrow/stream/247081-t-compiler.2Fperformance/topic/Slow.20Builtin.20Derives/near/215750815
Tomasz Miąsko [Thu, 12 Nov 2020 00:00:00 +0000 (00:00 +0000)]
Normalize function type during validation
During inlining, the callee body is normalized and has types revealed,
but some of locals corresponding to the arguments might come from the
caller body which is not. As a result the caller body does not pass
validation without additional normalization.
rustc_parse: Remove optimization for 0-length streams in `collect_tokens`
The optimization conflates empty token streams with unknown token stream, which is at least suspicious, and doesn't affect performance because 0-length token streams are very rare.
Tomasz Miąsko [Wed, 11 Nov 2020 00:00:00 +0000 (00:00 +0000)]
Never inline when `no_sanitize` attributes differ
The inliner looks if a sanitizer is enabled before considering
`no_sanitize` attribute as possible source of incompatibility.
The MIR inlining could happen in a crate with sanitizer disabled, but
code generation in a crate with sanitizer enabled, thus the attribute
would be incorrectly ignored.
To avoid the issue never inline functions with different `no_sanitize`
attributes.
Mara Bos [Thu, 12 Nov 2020 18:46:17 +0000 (19:46 +0100)]
Rollup merge of #78972 - ehuss:update-cargo, r=ehuss
Update cargo
5 commits in d5556aeb8405b1fe696adb6e297ad7a1f2989b62..8662ab427a8d6ad8047811cc4d78dbd20dd07699
2020-11-04 22:20:36 +0000 to 2020-11-12 03:47:53 +0000
- Check if rust-src contains a vendor dir, and patch it in (rust-lang/cargo#8834)
- Improve performance of almost fresh builds (rust-lang/cargo#8837)
- Use u32/64::to/from_le_bytes instead of bit fiddling (rust-lang/cargo#8847)
- Avoid constructing an anyhow::Error when not necessary (rust-lang/cargo#8844)
- Skip extracting .cargo-ok files from packages (rust-lang/cargo#8835)
Mara Bos [Thu, 12 Nov 2020 18:46:14 +0000 (19:46 +0100)]
Rollup merge of #78950 - khyperia:spirv-asm, r=Amanieu
Add asm register information for SPIR-V
As discussed in [zulip](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Defining.20asm!.20for.20new.20architecture), we at [rust-gpu](https://github.com/EmbarkStudios/rust-gpu) would like to support `asm!` for our SPIR-V backend. However, we cannot do so purely without frontend support: [this match](https://github.com/rust-lang/rust/blob/d4ea0b3e46a0303d5802b632e88ba1ba84d9d16f/compiler/rustc_target/src/asm/mod.rs#L185) fails and so `asm!` is not supported ([error reported here](https://github.com/rust-lang/rust/blob/d4ea0b3e46a0303d5802b632e88ba1ba84d9d16f/compiler/rustc_ast_lowering/src/expr.rs#L1095)). To resolve this, we need to stub out register information for SPIR-V to support getting the `asm!` content all the way to [`AsmBuilderMethods::codegen_inline_asm`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/traits/trait.AsmBuilderMethods.html#tymethod.codegen_inline_asm), at which point the rust-gpu backend can do all the parsing and codegen that is needed.
This is a pretty weird PR - adding support for a backend that isn't in-tree feels pretty gross to me, but I don't see an easy way around this. ``@Amanieu`` said I should submit it anyway, so, here we are! Let me know if this needs to go through a more formal process (MCP?) and what I should do to help this along.
I based this off the [wasm asm PR](https://github.com/rust-lang/rust/pull/78684), which unfortunately this PR conflicts with that one quite a bit, sorry for any merge conflict pain :(
---
Some open questions:
- What do we call the register class? Some context, SPIR-V is an SSA-based IR, there are "instructions" that create IDs (referred to as `<id>` in the spec), which can be referenced by other instructions. So, `reg` isn't exactly accurate, they're SSA IDs, not re-assignable registers.
- What happens when a SPIR-V register gets to the LLVM backend? Right now it's a `bug!`, but should that be a `sess.fatal()`? I'm not sure if it's even possible to reach that point, maybe there's a check that prevents the `spirv` target from even reaching that codepath.
Mara Bos [Thu, 12 Nov 2020 18:46:11 +0000 (19:46 +0100)]
Rollup merge of #78857 - SkiFire13:bheap-opt, r=KodrAus
Improve BinaryHeap performance
By changing the condition in the loops from `child < end` to `child < end - 1` we're guaranteed that `right = child + 1 < end` and since finding the index of the biggest sibling can be done with an arithmetic operation we can remove a branch from the loop body. The case where there's no right child, i.e. `child == end - 1` is instead handled outside the loop, after it ends; note that if the loops ends early we can use `return` instead of `break` since the check `child == end - 1` will surely fail.
I've also removed a call to `<[T]>::swap` that was hiding a bound check that [wasn't being optimized by LLVM](https://godbolt.org/z/zrhdGM).
A quick benchmarks on my pc shows that the gains are pretty significant:
Mara Bos [Thu, 12 Nov 2020 18:46:09 +0000 (19:46 +0100)]
Rollup merge of #78836 - fanzier:struct-and-slice-destructuring, r=petrochenkov
Implement destructuring assignment for structs and slices
This is the second step towards implementing destructuring assignment (RFC: rust-lang/rfcs#2909, tracking issue: #71126). This PR is the second part of #71156, which was split up to allow for easier review.
Note that the first PR (#78748) is not merged yet, so it is included as the first commit in this one. I thought this would allow the review to start earlier because I have some time this weekend to respond to reviews. If ``@petrochenkov`` prefers to wait until the first PR is merged, I totally understand, of course.
This PR implements destructuring assignment for (tuple) structs and slices. In order to do this, the following *parser change* was necessary: struct expressions are not required to have a base expression, i.e. `Struct { a: 1, .. }` becomes legal (in order to act like a struct pattern).
Unfortunately, this PR slightly regresses the diagnostics implemented in #77283. However, it is only a missing help message in `src/test/ui/issues/issue-77218.rs`. Other instances of this diagnostic are not affected. Since I don't exactly understand how this help message works and how to fix it yet, I was hoping it's OK to regress this temporarily and fix it in a follow-up PR.
Thanks to ``@varkor`` who helped with the implementation, particularly around the struct rest changes.
Joshua Nelson [Sun, 8 Nov 2020 16:45:13 +0000 (11:45 -0500)]
Don't reuse bindings for `ref mut`
Reusing bindings causes errors later in lowering:
```
error[E0596]: cannot borrow `vec` as mutable, as it is not declared as mutable
--> /checkout/src/test/ui/async-await/argument-patterns.rs:12:20
|
LL | async fn b(n: u32, ref mut vec: A) {
| ^^^^^^^^^^^
| |
| cannot borrow as mutable
| help: consider changing this to be mutable: `mut vec`
```
bors [Thu, 12 Nov 2020 15:34:09 +0000 (15:34 +0000)]
Auto merge of #76256 - tgnottingham:issue-74890, r=nikomatsakis
incr-comp: hash and serialize span end line/column
Hash both the length and the end location (line/column) of a span. If we
hash only the length, for example, then two otherwise equal spans with
different end locations will have the same hash. This can cause a
problem during incremental compilation wherein a previous result for a
query that depends on the end location of a span will be incorrectly
reused when the end location of the span it depends on has changed. A
similar analysis applies if some query depends specifically on the
length of the span, but we only hash the end location. So hash both.
I haven't thought about what other problems this root cause could be responsible for. Please let me know if anything springs to mind. I believe the issue has existed since the inception of incremental compilation.
DevJPM [Sat, 7 Nov 2020 22:25:45 +0000 (23:25 +0100)]
fully exploited the dropped support of LLVM 8
This commit grepped for LLVM_VERSION_GE, LLVM_VERSION_LT, get_major_version and
min-llvm-version and statically evaluated every expression possible
(and sensible) assuming that the LLVM version is >=9 now
DevJPM [Sat, 7 Nov 2020 20:25:42 +0000 (21:25 +0100)]
explicitly add llvm-9-dev in dockerfile
apparently llvm-8-tools already had llvm-8-dev as a dependency
which was removed in llvm-9-tools, so we need to explicitly pull
llvm-9-dev to make a build