bors [Wed, 12 Jun 2019 04:57:35 +0000 (04:57 +0000)]
Auto merge of #61758 - Centril:rollup-ew2uxng, r=Centril
Rollup of 9 pull requests
Successful merges:
- #60187 (Generator optimization: Overlap locals that never have storage live at the same time)
- #61348 (Implement Clone::clone_from for Option and Result)
- #61568 (Use Symbol, Span in libfmt_macros)
- #61632 (ci: Collect CPU usage statistics on Azure)
- #61654 (use pattern matching for slices destructuring)
- #61671 (implement nth_back for Range(Inclusive))
- #61688 (is_fp and is_floating_point do the same thing, remove the former)
- #61705 (Pass cflags rather than cxxflags to LLVM as CMAKE_C_FLAGS)
- #61734 (Migrate rust-by-example to MdBook2)
Rollup merge of #61632 - alexcrichton:azure-pipelines-cpu, r=pietroalbini
ci: Collect CPU usage statistics on Azure
This commit adds a script which we'll execute on Azure Pipelines which
is intended to run in the background and passively collect CPU usage
statistics for our builders. The intention here is that we can use this
information over time to diagnose issues with builders, see where we can
optimize our build, fix parallelism issues, etc. This might not end up
being too useful in the long run but it's data we've wanted to collect
for quite some time now, so here's a stab at it!
Comments about how this is intended to work can be found in the python
script used here to collect CPU usage statistics.
Rollup merge of #61568 - Mark-Simulacrum:symbol-fmt-macros, r=estebank
Use Symbol, Span in libfmt_macros
I'm not super happy with this, personally, but I think it might be a decent start -- happy to take suggestions as to how to expand this or change things further.
Rollup merge of #60187 - tmandry:generator-optimization, r=eddyb
Generator optimization: Overlap locals that never have storage live at the same time
The specific goal of this optimization is to optimize async fns which use `await!`. Notably, `await!` has an enclosing scope around the futures it awaits ([definition](https://github.com/rust-lang/rust/blob/08bfe16129b0621bc90184f8704523d4929695ef/src/libstd/macros.rs#L365-L381)), which we rely on to implement the optimization.
More generally, the optimization allows overlapping the storage of some locals which are never storage-live at the same time. **We care about storage-liveness when computing the layout, because knowing a field is `StorageDead` is the only way to prove it will not be accessed, either directly or through a reference.**
To determine whether we can overlap two locals in the generator layout, we look at whether they might *both* be `StorageLive` at any point in the MIR. We use the `MaybeStorageLive` dataflow analysis for this. We iterate over every location in the MIR, and build a bitset for each local of the locals it might potentially conflict with.
Next, we assign every saved local to one or more variants. The variants correspond to suspension points, and we include the set of locals live across a given suspension point in the variant. (Note that we use liveness instead of storage-liveness here; this ensures that the local has actually been initialized in each variant it has been included in. If the local is not live across a suspension point, then it doesn't need to be included in that variant.). It's important to note that the variants are a "view" into our layout.
For the layout computation, we use a simplified approach.
1. Start with the set of locals assigned to only one variant. The rest are disqualified.
2. For each pair of locals which may conflict *and are not assigned to the same variant*, we pick one local to disqualify from overlapping.
Disqualified locals go into a non-overlapping "prefix" at the beginning of our layout. This means they always have space reserved for them. All the locals that are allowed to overlap in each variant are then laid out after this prefix, in the "overlap zone".
So, if A and B were disqualified, and X, Y, and Z were all eligible for overlap, our generator might look something like this:
You can think of a generator as an enum, where some fields are shared between variants. e.g.
where every mention of `A` and `B` refer to the same field, which does not move when changing variants. Note that `A` and `B` would automatically be sent to the prefix in this example. Assuming that `X` is never `StorageLive` at the same time as either `Y` or `Z`, it would be allowed to overlap with them.
Note that if two locals (`Y` and `Z` in this case) are assigned to the same variant in our generator, their memory would never overlap in the layout. Thus they can both be eligible for the overlapping section, even if they are storage-live at the same time.
---
Depends on:
- [x] #59897 Multi-variant layouts for generators
- [x] #60840 Preserve local scopes in generator MIR
- [x] #61373 Emit StorageDead along unwind paths for generators
Before merging:
- [x] ~Wrap the types of all generator fields in `MaybeUninitialized` in layout::ty::field~ (opened #60889)
- [x] Make PR description more complete (e.g. explain why storage liveness is important and why we have to check every location)
- [x] Clean up TODO
- [x] Fix the layout code to enforce that the same field never moves around in the generator
- [x] Add tests for async/await
- [x] ~Reduce # bits we store by half, since the conflict relation is symmetric~ (note: decided not to do this, for simplicity)
- [x] Store liveness information for each yield point in our `GeneratorLayout`, that way we can emit more useful debuginfo AND tell miri which fields are definitely initialized for a given variant (see discussion at https://github.com/rust-lang/rust/pull/59897#issuecomment-489468627)
bors [Wed, 12 Jun 2019 02:13:11 +0000 (02:13 +0000)]
Auto merge of #60187 - tmandry:generator-optimization, r=eddyb
Generator optimization: Overlap locals that never have storage live at the same time
The specific goal of this optimization is to optimize async fns which use `await!`. Notably, `await!` has an enclosing scope around the futures it awaits ([definition](https://github.com/rust-lang/rust/blob/08bfe16129b0621bc90184f8704523d4929695ef/src/libstd/macros.rs#L365-L381)), which we rely on to implement the optimization.
More generally, the optimization allows overlapping the storage of some locals which are never storage-live at the same time. **We care about storage-liveness when computing the layout, because knowing a field is `StorageDead` is the only way to prove it will not be accessed, either directly or through a reference.**
To determine whether we can overlap two locals in the generator layout, we look at whether they might *both* be `StorageLive` at any point in the MIR. We use the `MaybeStorageLive` dataflow analysis for this. We iterate over every location in the MIR, and build a bitset for each local of the locals it might potentially conflict with.
Next, we assign every saved local to one or more variants. The variants correspond to suspension points, and we include the set of locals live across a given suspension point in the variant. (Note that we use liveness instead of storage-liveness here; this ensures that the local has actually been initialized in each variant it has been included in. If the local is not live across a suspension point, then it doesn't need to be included in that variant.). It's important to note that the variants are a "view" into our layout.
For the layout computation, we use a simplified approach.
1. Start with the set of locals assigned to only one variant. The rest are disqualified.
2. For each pair of locals which may conflict *and are not assigned to the same variant*, we pick one local to disqualify from overlapping.
Disqualified locals go into a non-overlapping "prefix" at the beginning of our layout. This means they always have space reserved for them. All the locals that are allowed to overlap in each variant are then laid out after this prefix, in the "overlap zone".
So, if A and B were disqualified, and X, Y, and Z were all eligible for overlap, our generator might look something like this:
You can think of a generator as an enum, where some fields are shared between variants. e.g.
where every mention of `A` and `B` refer to the same field, which does not move when changing variants. Note that `A` and `B` would automatically be sent to the prefix in this example. Assuming that `X` is never `StorageLive` at the same time as either `Y` or `Z`, it would be allowed to overlap with them.
Note that if two locals (`Y` and `Z` in this case) are assigned to the same variant in our generator, their memory would never overlap in the layout. Thus they can both be eligible for the overlapping section, even if they are storage-live at the same time.
---
Depends on:
- [x] #59897 Multi-variant layouts for generators
- [x] #60840 Preserve local scopes in generator MIR
- [x] #61373 Emit StorageDead along unwind paths for generators
Before merging:
- [x] ~Wrap the types of all generator fields in `MaybeUninitialized` in layout::ty::field~ (opened #60889)
- [x] Make PR description more complete (e.g. explain why storage liveness is important and why we have to check every location)
- [x] Clean up TODO
- [x] Fix the layout code to enforce that the same field never moves around in the generator
- [x] Add tests for async/await
- [x] ~Reduce # bits we store by half, since the conflict relation is symmetric~ (note: decided not to do this, for simplicity)
- [x] Store liveness information for each yield point in our `GeneratorLayout`, that way we can emit more useful debuginfo AND tell miri which fields are definitely initialized for a given variant (see discussion at https://github.com/rust-lang/rust/pull/59897#issuecomment-489468627)
bors [Tue, 11 Jun 2019 20:45:17 +0000 (20:45 +0000)]
Auto merge of #61735 - eddyb:must-use-life, r=oli-obk
Add deny(unused_lifetimes) to all the crates that have deny(internal).
@Zoxc brought up, regarding #61722, that we don't force the removal of unused lifetimes.
Turns out that it's not that bad to enable for compiler crates (I wonder why it's not `warn` by default?).
I would've liked to enable `single_use_lifetimes` as well, but https://github.com/rust-lang/rust/issues/53738 makes it unusable for now.
For the `rustfmt` commit, I used https://github.com/rust-lang/rustfmt/issues/1324#issuecomment-482109952, and manually filtered out some noise.
Rollup merge of #61606 - petrochenkov:legclean, r=pnkfelix
Remove some legacy proc macro flavors
Namely
- `IdentTT` (`foo! ident { ... }`). Can be replaced with `foo! { ident ... }` or something similar.
- `MultiDecorator`. Can be replaced by `MultiModifier` (aka `LegacyAttr` after renaming).
- `DeclMacro`. It was a less powerful duplicate of `NormalTT` (aka `LegacyBang` after renaming) and can be replaced by it.
Stuff like this slows down any attempts to refactor the expansion infra, so it's desirable to retire it already.
I'm not sure whether a lang team decision is necessary, but would be nice to land this sooner because I have some further work in this area scheduled.
The documentation commit (https://github.com/rust-lang/rust/commit/a9397fd0d5eede4bbc0ada94bf92657ca8084cb3) describes how the remaining variants are different from each other and shows that there's actually some system behind them.
The last commit renames variants of `SyntaxExtension` in more systematic way.
- `ProcMacro` -> `Bang`
- `NormalTT` -> `LegacyBang`
- `AttrProcMacro` -> `Attr`
- `MultiModifier` -> `LegacyAttr`
- `ProcMacroDerive` -> `Derive`
- `BuiltinDerive` -> `LegacyDerive`
All the `Legacy*` variants are AST-based, as opposed to "modern" token-based variants.
Alex Crichton [Fri, 7 Jun 2019 18:30:11 +0000 (11:30 -0700)]
ci: Collect CPU usage statistics on Azure
This commit adds a script which we'll execute on Azure Pipelines which
is intended to run in the background and passively collect CPU usage
statistics for our builders. The intention here is that we can use this
information over time to diagnose issues with builders, see where we can
optimize our build, fix parallelism issues, etc. This might not end up
being too useful in the long run but it's data we've wanted to collect
for quite some time now, so here's a stab at it!
Comments about how this is intended to work can be found in the python
script used here to collect CPU usage statistics.
bors [Mon, 10 Jun 2019 23:32:12 +0000 (23:32 +0000)]
Auto merge of #60793 - Xanewok:raw-string-cleanup, r=petrochenkov
lexer: Disallow bare CR in raw byte strings
Handles bare CR ~but doesn't translate `\r\n` to `\n` yet in raw strings yet~ and translates CRLF to LF in raw strings.
As a side-note I think it'd be good to change the `unescape_` to return plain iterators to reduce some boilerplate (e.g. `has_error` could benefit from collecting `Result<T>` and aborting early on errors) but will do that separately, unless I missed something here that prevents it.
David Wood [Sun, 9 Jun 2019 19:54:28 +0000 (20:54 +0100)]
typeck: Fix const generic in repeat param ICE.
This commit fixes an ICE that occured when a const generic was used in
a repeat expression. This was due to the code expecting the length of
the repeat expression to be const evaluatable to a constant, but a const
generic parameter is not (however, it can be made into a constant).
bors [Mon, 10 Jun 2019 11:15:07 +0000 (11:15 +0000)]
Auto merge of #61716 - Centril:rollup-nxwf5ol, r=Centril
Rollup of 5 pull requests
Successful merges:
- #59600 (Replaced linear token counting macros with optimized implementation)
- #61501 (get rid of real_intrinsics module)
- #61570 (Fix issues with const argument inference)
- #61683 (Haiku: the maximum stack size is 16 MB)
- #61697 (submodules: update clippy from 71be6f62 to c0dbd34b)
Rollup merge of #61683 - nielx:haiku-stack-limit, r=nagisa
Haiku: the maximum stack size is 16 MB
This keeps the compiler from crashing every time it is invoked. No functional change on other platforms.
This patch is similar to the limitation that is in the [librustdoc/lib.rs](https://github.com/rust-lang/rust/blob/57e13e0325c1d41161a31de1f8520538ec2c575c/src/librustdoc/lib.rs#L89).
Replaced linear token counting macros with optimized implementation
There are currently two distinct token-counting macros in the source. Both implement the trivial algorithm, with linear complexity. They may or may not be adequate for their use case, but considering that other people are probably going to copy and paste them whenever they need a token-counting macro, I replaced them with an optimized implementation with logarithmic complexity.
When one tries to create a thread with a requested stack size larger
than 16 MB, the call will fail and the compiler will bail out. Therefore
we should limit the size of the thread stack to 16 MB on Haiku.
bors [Mon, 10 Jun 2019 02:35:29 +0000 (02:35 +0000)]
Auto merge of #61706 - petrhosek:bootstrap-cp-r, r=Mark-Simulacrum
Use Build::read_dir instead of fs::read_dir in Build::cp_r
Build::read_dir does better error handling when the directory doesn't
exist; it actually prints the name of the directory rather than just
printing the underlying error "No such file or directory" which on
its own isn't very useful.
Petr Hosek [Sun, 9 Jun 2019 23:57:17 +0000 (16:57 -0700)]
Use Build::read_dir instead of fs::read_dir in Build::cp_r
Build::read_dir does better error handling when the directory doesn't
exist; it actually prints the name of the directory rather than just
printing the underlying error "No such file or directory" which on
its own isn't very useful.
Mark Rousskov [Tue, 4 Jun 2019 15:03:43 +0000 (09:03 -0600)]
Introduce InnerSpan abstraction
This should be used when trying to get at subsets of a larger span,
especially when the larger span is not available in the code attempting
to work with those subsets (especially common in the fmt_macros crate).
This is usually a good replacement for (BytePos, BytePos) and (usize,
usize) tuples.
This commit also removes from_inner_byte_pos, since it took usize
arguments, which is error prone.
projects / rust.git / log