Auto merge of #61268 - michaelwoerister:stabilize-pgo, r=alexcrichton
Stabilize support for Profile-guided Optimization
This PR makes profile-guided optimization available via the `-C profile-generate` / `-C profile-use` pair of commandline flags and adds end-user documentation for the feature to the [rustc book](https://doc.rust-lang.org/rustc/). The PR thus ticks the last two remaining checkboxes of the [stabilization tracking issue](https://github.com/rust-lang/rust/issues/59913).
From the tracking issue:
> Profile-guided optimization (PGO) is a common optimization technique for ahead-of-time compilers. It works by collecting data about a program's typical execution (e.g. probability of branches taken, typical runtime values of variables, etc) and then uses this information during program optimization for things like inlining decisions, machine code layout, or indirect call promotion.
If you are curious about how this can be used, there is a rendered version of the documentation this PR adds available [here](
https://github.com/michaelwoerister/rust/blob/stabilize-pgo/src/doc/rustc/src/profile-guided-optimization.md).
Auto merge of #61922 - tmandry:moar-generator-optimization, r=matthewjasper
Don't store locals that have been moved from in generators
This avoids reserving storage in generators for locals that are moved
out of (and not re-initialized) prior to yield points. Fixes #59123.
This adds a new dataflow analysis, `RequiresStorage`, to determine whether the storage of a local can be destroyed without being observed by the program. The rules are:
1. StorageLive(x) => mark x live
2. StorageDead(x) => mark x dead
3. If a local is moved from, _and has never had its address taken_, mark it dead
4. If (any part of) a local is initialized, mark it live'
This is used to determine whether to save a local in the generator object at all, as well as which locals can be overlapped in the generator layout.
Here's the size in bytes of all testcases included in the change, before and after the change:
async fn test |Size before |Size after
-----------------|------------|----------
single | 1028 | 1028
single_with_noop | 2056 | 1032
joined | 5132 | 3084
joined_with_noop | 8208 | 3084
generator test |Size before |Size after
----------------------------|------------|----------
move_before_yield | 1028 | 1028
move_before_yield_with_noop | 2056 | 1032
overlap_move_points | 3080 | 2056
## Future work
Note that there is a possible extension to this optimization, which modifies rule 3 to read: "If a local is moved from, _**and either has never had its address taken, or is Freeze and has never been mutably borrowed**_, mark it dead." This was discussed at length in #59123 and then #61849. Because this would cause some behavior to be UB which was not UB before, it's a step that needs to be taken carefully.
A more immediate priority for me is inlining `std::mem::size_of_val(&x)` so it becomes apparent that the address of `x` is not taken. This way, using `size_of_val` to look at the size of your inner futures does not affect the size of your outer future.
cc @cramertj @eddyb @Matthias247 @nikomatsakis @RalfJung @Zoxc
Auto merge of #62279 - pietroalbini:fix-azure-crlf, r=alexcrichton
ci: explicitly disable CRLF conversion on Windows
The Azure image enables CRLF conversion on Windows builders, but that caused regressions both in our test suite (the miri test suite broke) and in the ecosystem, since we started shipping install scripts with CRLF endings instead of the old LF. The [Godbolt Compiler Explorer](https://godbolt.org/) is one such case of breakage.
This adds a step to the build explicitly disabling the conversion before the repository is checked out.
Pietro Albini [Mon, 1 Jul 2019 19:57:08 +0000 (21:57 +0200)]
ci: explicitly disable CRLF conversion on Windows
The Azure image enables CRLF conversion on Windows builders, but that
caused regressions both in our test suite (the miri test suite broke)
and in the ecosystem, since we started shipping install scripts with
CRLF endings instead of the old LF. The Godbolt Compiler Explorer is one
such case of breakage.
This adds a step to the build explicitly disabling the conversion before
the repository is checked out.
Igor Matuszewski [Sun, 30 Jun 2019 23:36:43 +0000 (01:36 +0200)]
Update RLS to disable spurious client_find_definitions test
Since switching CI to Azure Pipelines it seems that this test seems
to fail more consistently, so let's disable that for now. It helps
that we have less than a week before release - we disallow PRs that
break the tools to land in this period, so this makes landing critical
PRs smoother now.
Auto merge of #62253 - Centril:rollup-115uuuq, r=Centril
Rollup of 8 pull requests
Successful merges:
- #62062 (Use a more efficient iteration order for forward dataflow)
- #62063 (Use a more efficient iteration order for backward dataflow)
- #62224 (rustdoc: remove unused derives and variants)
- #62228 (Extend the #[must_use] lint to boxed types)
- #62235 (Extend the `#[must_use]` lint to arrays)
- #62239 (Fix a typo)
- #62241 (Always parse 'async unsafe fn' + properly ban in 2015)
- #62248 (before_exec actually will only get deprecated with 1.37)
Auto merge of #61682 - Centril:stabilize-type_alias_enum_variants, r=petrochenkov
Stabilize `type_alias_enum_variants` in Rust 1.37.0
Stabilize `#![feature(type_alias_enum_variants)]` which allows type-relative resolution with highest priority to `enum` variants in both expression and pattern contexts. For example, you may now write:
```rust
enum Option<T> {
None,
Some(T),
}
type OptAlias<T> = Option<T>;
fn work_on_alias(x: Option<u8>) -> u8 {
match x {
OptAlias::Some(y) => y + 1,
OptAlias::None => 0,
}
}
```
Rollup merge of #62241 - Centril:fix-async-unsafe-order, r=petrochenkov
Always parse 'async unsafe fn' + properly ban in 2015
Parse `async unsafe fn` not `unsafe async fn` in implementations. We also take the opportunity to properly ban `async fn` in Rust 2015 when they are inside implementations.
Rollup merge of #62239 - lcolaholicl:lcolaholicl-patch-1, r=kennytm
Fix a typo
The definition of 京 seems to be capital, but not capitol.
[reference](https://en.wiktionary.org/wiki/%E4%BA%AC#Etymology_1)
[another reference](https://jisho.org/word/%E4%BA%AC)
Rollup merge of #62224 - euclio:remove-derives, r=GuillaumeGomez
rustdoc: remove unused derives and variants
Though many structs in rustdoc derive `RustcEncodable` and `RustcDecodable`, the impls do not appear to be used by the crate or its dependents. Removing them revealed some enum variants that are never constructed, too.
Rollup merge of #62063 - ecstatic-morse:dataflow-backward-order, r=nagisa
Use a more efficient iteration order for backward dataflow
This applies the same basic principle as #62062 to the reverse dataflow analysis used to compute liveness information. It is functionally equivalent, except that post-order is used instead of reverse post-order.
In the long-term, `BitDenotation` should probably be extended to support both forward and backward dataflow, but there's some more work needed to get to that point.
Rollup merge of #62062 - ecstatic-morse:dataflow-order, r=nagisa
Use a more efficient iteration order for forward dataflow
Currently, dataflow begins by visiting each block in order of ID (`BasicBlock(0)`, `BasicBlock(1)`, etc.). This PR changes that initial iteration to reverse post-order (see [this blog post](https://eli.thegreenplace.net/2015/directed-graph-traversal-orderings-and-applications-to-data-flow-analysis/#data-flow-analysis) for more info). This ensures that the effects of all predecessors will be applied before a basic block is visited if the CFG has no back-edges, and should result in less total iterations even when back-edges exist. This should not change the results of dataflow analysis.
The current ordering for basic blocks may be pretty close to RPO already--`BasicBlock(0)` is already the start block--in which case the cost of doing the traversal up front will outweigh the efficiency gains.
A perf run is needed to check this.
bors [Sun, 30 Jun 2019 12:15:14 +0000 (12:15 +0000)]
Auto merge of #59722 - Zoxc:query-cache, r=eddyb
Clean up query cache code
There are a couple of queries for which we do not promote saved results, which have been removed from caching here. This was likely the cause of the regression in https://github.com/rust-lang/rust/pull/57293#issuecomment-476421298.
bors [Sun, 30 Jun 2019 08:42:58 +0000 (08:42 +0000)]
Auto merge of #61252 - indygreg:upgrade-musl, r=alexcrichton
Update musl to 1.1.22
This is the latest available version. I noticed Rust wasn't using the
latest version when I attempted to have Cargo link object files produced
outside of Cargo / Rust's toolchain and was getting missing symbol
errors. Those missing symbols were added in 1.1.21 and 1.1.22.
I'm not fully sure of the implications of this change or how
comprehensive the test coverage is. Upstream changes in 1.1.21 and
1.1.22 can be found at
https://git.musl-libc.org/cgit/musl/tree/WHATSNEW?h=v1.1.22&id=e97681d6f2c44bf5fa9ecdd30607cb63c780062e#n1989.
bors [Sat, 29 Jun 2019 11:45:11 +0000 (11:45 +0000)]
Auto merge of #62226 - Centril:rollup-rcy1alx, r=Centril
Rollup of 7 pull requests
Successful merges:
- #61199 (Revert "Set test flag when rustdoc is running with --test option" )
- #61755 (Add `--pass $mode` to compiletest through `./x.py`)
- #61818 (Issue #60709 test)
- #62023 (publish_toolstate: don't use 'new' from inside the loop)
- #62104 (Inform the query system about properties of queries at compile time)
- #62163 (Avoid mem::uninitialized() in std::sys::unix)
- #62204 (doc(libcore) Fix CS)
Rollup merge of #62023 - RalfJung:miri-toolstate, r=kennytm
publish_toolstate: don't use 'new' from inside the loop
I think I made a mistake in https://github.com/rust-lang/rust/pull/61938 by using `new` outside the inner loop. This PR fixes that.
However, no issue got created at all for https://github.com/rust-lang/rust/pull/62003#issuecomment-504356964, and I don't know why that is. The script should be [triggered by Traivs](https://github.com/rust-lang/rust/blob/56a12b2ad058f22f1ef090713df15598525ba4a4/.travis.yml#L240), and I can find no trace of that in [the build log](https://travis-ci.com/rust-lang/rust/jobs/209880042).
Rollup merge of #61755 - Centril:compiletest-force-check, r=petrochenkov
Add `--pass $mode` to compiletest through `./x.py`
Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`.
When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file.
The modes are explained in https://github.com/rust-lang/rust/pull/61778:
- `check` has the same effect as `cargo check`
- `build` or `compile` have the same effect as `cargo build`
- `run` has the same effect as `cargo run`
On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`.
bors [Fri, 28 Jun 2019 14:08:07 +0000 (14:08 +0000)]
Auto merge of #62192 - Centril:rollup-kssnlta, r=Centril
Rollup of 16 pull requests
Successful merges:
- #61878 (improve pinning projection docs)
- #62043 (Remove `FnBox`)
- #62067 (Add suggestion for missing `.await` keyword)
- #62076 (Updated RELEASES.md for 1.36.0)
- #62102 (call out explicitly that general read needs to be called with an initialized buffer)
- #62106 (Add more tests for async/await)
- #62124 (refactor lexer to use idiomatic borrowing)
- #62131 (libsyntax: Fix some Clippy warnings)
- #62152 (Don't ICE on item in `.await` expression)
- #62154 (Remove old fixme)
- #62155 (Add regression test for MIR drop generation in async loops)
- #62156 (Update books)
- #62160 (Remove outdated question_mark_macro_sep lint)
- #62164 (save-analysis: use buffered writes)
- #62171 (rustc: Retry SIGILL linker invocations)
- #62176 (Update RLS)
bors [Fri, 28 Jun 2019 10:20:36 +0000 (10:20 +0000)]
Auto merge of #62179 - alexcrichton:less-travis-appveyor, r=pietroalbini
ci: Move most builders to Azure Pipelines
This commit disables all builders on Travis and almost all builders on
AppVeyor now that they're all running on Azure Pipelines. There is one
remaining builder on AppVeyor which hasn't been migrated yet due to a
test failure on Azure, which we'll be debugging soon. One remaining
builder is left on Travis which is the tools builder whenever a
submodule is changed, but we'll probably turn that off soon since it's
just for PRs.
The other major change in this PR is that the auto builders on Azure are
now configured with "real" prod credentials which should cause them to
publish all artifacts into the official CI buckets.
Alex Crichton [Thu, 27 Jun 2019 12:30:57 +0000 (05:30 -0700)]
ci: Move most builders to Azure Pipelines
This commit disables all builders on Travis and almost all builders on
AppVeyor now that they're all running on Azure Pipelines. There is one
remaining builder on AppVeyor which hasn't been migrated yet due to a
test failure on Azure, which we'll be debugging soon. One remaining
builder is left on Travis which is the tools builder whenever a
submodule is changed, but we'll probably turn that off soon since it's
just for PRs.
The other major change in this PR is that the auto builders on Azure are
now configured with "real" prod credentials which should cause them to
publish all artifacts into the official CI buckets.
Rollup merge of #62176 - Xanewok:update-rls, r=alexcrichton
Update RLS
Merged PRs:
* fix(cmd): make clear_env_rust_log default to false (https://github.com/rust-lang/rls/pull/1486)
- Retain `RUST_LOG` in `rls --cli` mode
* Pass --file-lines to rustfmt only if specified (https://github.com/rust-lang/rls/pull/1497)
- Fix entire-file formatting when using external rustfmt (specified via `rustfmt_path` config)
* Ensure that --error-format is only passed once to `rustc` (https://github.com/rust-lang/rls/pull/1500)
- Unbreaks RLS when used together with Cargo [pipelining build](https://github.com/rust-lang/rust/issues/60988) feature (@alexcrichton I'd consider this a stabilization blocker, mind adding it to the tracking issue for the record? :bowing_man: )
Rollup merge of #62171 - alexcrichton:more-restarts, r=eddyb
rustc: Retry SIGILL linker invocations
We've seen quite a few issues with spurious illegal instructions getting
executed on OSX on CI recently. For whatever reason `cc` itself is
executing an illegal instruction and we're not really getting any other
information about what's happening. Since we're already retrying the
linker when it segfaults, let's just continue to retry everything and
automatically reinvoke the linker when it fails with an illegal instruction.
2 commits in 08ae27a4921ca53967656a7391c82f6c0ddd1ccc..7a5aab5fd50d6290679beb4cf42fa5f46ed22aec
2019-06-17 11:24:13 -0700 to 2019-06-20 17:38:52 +0200
- Update for cfg on generic parameter. (rust-lang-nursery/reference#624)
- Link to "const functions" section from constant expressions list (rust-lang-nursery/reference#623)
2 commits in b27472962986e85c94f4183b1a6d2207660d3ed6..62b3ff2cb44dd8b648c3ef2a9347c3706d148014
2019-06-17 15:52:07 -0300 to 2019-06-24 09:17:21 -0300
- A couple of fixes for the `Box, stack and heap` chapter. (rust-lang/rust-by-example#1206)
- [typo] Note, that it is -> Note that, it is (rust-lang/rust-by-example#1207)
## rustc-guide
38 commits in f55e97c145cf37fd664db2e0e2f2d05df328bf4f..abf512fc9cc969dcbea69aa15b44586bbeb13c2d
2019-06-15 17:29:12 -0500 to 2019-06-26 11:05:58 -0500
- fix long line
- add `point` to the glossary and link a use of it
- fix indentation
- Update src/borrow_check/region_inference/placeholders_and_universes.md
- Update src/borrow_check/region_inference/placeholders_and_universes.md
- Update src/borrow_check/region_inference/placeholders_and_universes.md
- Update src/borrow_check/region_inference/member_constraints.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/member_constraints.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/member_constraints.md
- Update src/borrow_check/region_inference/member_constraints.md
- Update src/borrow_check/region_inference/lifetime_parameters.md
- Update src/borrow_check/region_inference/member_constraints.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- Update src/borrow_check/region_inference/lifetime_parameters.md
- Update src/borrow_check/region_inference/lifetime_parameters.md
- Update src/borrow_check/region_inference/lifetime_parameters.md
- Update src/borrow_check/region_inference/constraint_propagation.md
- adjust overview slightly
- describe region inference and member constraints in some detail
- start filling out the constraint propagation chapter in more detail
- break out parts of the region inference chapter into sub-chapters
- fix typo
- avoid ftp links
- fix broken links
- add bibligraphy appendix
- Update to mdbook-linkcheck 0.3.0
- Update mdbook
- Change stage0 cfg_attr to bootstrap
- fix compiler-team
- Added Rustc Debugger Support Chapter
Rollup merge of #62152 - doctorn:async_let_ice, r=cramertj
Don't ICE on item in `.await` expression
The code for lowering a `.await` expression missed that item IDs may already have been assigned for items inside of an `async` block, or for closures. This change means we no longer exit early after finding a `.await` in a block that isn't `async` and instead just emit the error. This avoids an ICE generated due to item IDs not being densely generated. (The `YieldSource` of the generated `yield` expression is used to avoid errors generated about having `yield` expressions outside of generator literals.)
Rollup merge of #62131 - Xanewok:clip-some-nits, r=petrochenkov
libsyntax: Fix some Clippy warnings
When I was working on it before a lot of these popped up in the RLS so I figured I'll send a small patch fixing only the (hopefully) uncontroversial ones.
Others that could be also fixed included also [`clippy::print_with_newline`](https://rust-lang.github.io/rust-clippy/master/index.html#print_with_newline) and [`clippy::cast_lossless`](https://rust-lang.github.io/rust-clippy/master/index.html#cast_lossless). Should I add them as well?
since most of it touches libsyntax...
r? @petrochenkov
Rollup merge of #62067 - doctorn:await_diagnostic, r=matthewjasper
Add suggestion for missing `.await` keyword
This commit adds a suggestion diagnostic for missing `.await`. In order to do this, the trait `Future` is promoted to a lang item.
Compiling code of the form:
```rust
#![feature(async_await)]
fn take_u32(x: u32) {}
async fn make_u32() -> u32 {
22
}
async fn foo() {
let x = make_u32();
take_u32(x)
}
fn main() {}
```
Will now result in the suggestion:
```
error[E0308]: mismatched types
--> src/main.rs:11:18
|
11 | take_u32(x)
| ^
| |
| expected u32, found opaque type
| help: consider using `.await` here: `x.await`
|
= note: expected type `u32`
found type `impl std::future::Future`
```
This commit does not cover chained expressions and therefore only covers the case originally pointed out in #61076. Cases I can think of that still need to be covered:
- [ ] Return places for functions
- [ ] Field access
- [ ] Method invocation
I'm planning to submit PRs for each of these separately as and when I have figured them out.
Rollup merge of #61878 - RalfJung:pin, r=Dylan-DPC
improve pinning projection docs
This tries to improve the explanation of structural pinning and pinning projections based on [this URLO thread](https://users.rust-lang.org/t/when-is-it-safe-to-move-a-member-value-out-of-a-pinned-future/28182).
Dylan MacKenzie [Sat, 22 Jun 2019 20:27:16 +0000 (13:27 -0700)]
Use more efficient iteration order for backward dataflow
This applies the same basic principle as #62062 to the reverse dataflow
analysis used to compute liveness information. It is functionally
equivalent, except that post-order is used instead of reverse post-order.
Some `mir::Body`s contain basic blocks which are not reachable from the
`START_BLOCK`. We need to add them to the work queue as well to preserve
the original semantics.
Dylan MacKenzie [Sat, 22 Jun 2019 20:00:17 +0000 (13:00 -0700)]
Use a more efficient iteration order for forward dataflow
Currently, dataflow begins by visiting each block in order of ID
(`BasicBlock(0)`, `BasicBlock(1)`, etc.). This PR changes that initial
iteration to reverse post-order. This ensures that the effects of all
predecessors will be applied before a basic block is visited if the CFG
has no back-edges, and should result in less total iterations even when
back-edges exist. This should not change the results of dataflow
analysis.
The current ordering for basic blocks is pretty close to RPO
already--`BasicBlock(0)` is already the start block, so the gains from
this are pretty small, especially since we need to do an extra traversal
up front.
Note that some basic blocks are unreachable from the `START_BLOCK`
during dataflow. We add these blocks to the work queue as well to
preserve the original behavior.
Alex Crichton [Thu, 27 Jun 2019 09:14:57 +0000 (02:14 -0700)]
rustc: Retry SIGILL linker invocations
We've seen quite a few issues with spurious illegal instructions getting
executed on OSX on CI recently. For whatever reason `cc` itself is
executing an illegal instruction and we're not really getting any other
information about what's happening. Since we're already retrying the
linker when it segfaults, let's just continue to retry everything and
automatically reinvoke the linker when it fails with an illegal instruction.
Rollup merge of #62145 - alexcrichton:sync-azure, r=pietroalbini
ci: Sync AppVeyor/Travis with Azure configuration
Manually make sure that we do the same thing across all the services,
uncovering one spot where we needed to pass one more configure flag on
Azure but otherwise we're good to go!
Rollup merge of #62143 - alexcrichton:toolstate, r=pietroalbini
ci: Publish toolstate changes from Azure
This commit moves toolstate publishing from Travis to Azure. We've been
testing on azure for some time now and this works by deleting the Travis
config and updating the credentials used on Azure.