bors [Sun, 8 Nov 2020 20:00:51 +0000 (20:00 +0000)]
Auto merge of #78712 - petrochenkov:visitok, r=Aaron1011
rustc_ast: Visit tokens stored in AST nodes in mutable visitor
After #77271 token visiting is enabled only for one visitor in `rustc_expand\src\mbe\transcribe.rs` which applies hygiene marks to tokens produced by declarative macros (`macro_rules` or `macro`), so this change doesn't affect anything else.
When a macro has some interpolated token from an outer macro in its output
```rust
macro inner() {
$interpolated
}
```
we can use the usual interpretation of interpolated tokens in token-based model - a None-delimited group - to write this macro in an equivalent form
```rust
macro inner() {
⟪ a b c d ⟫
}
```
When we are expanding the macro `inner` we need to apply hygiene marks to all tokens produced by it, including the tokens inside the group.
Before this PR we did this by visiting the AST piece inside the interpolated token and applying marks to all spans in it.
I'm not sure this is 100% correct (ideally we should apply the marks to tokens and then re-parse the AST from tokens), but it's a very good approximation at least.
We didn't however apply the marks to actual tokens stored in the nonterminal, so if we used the nonterminal as a token rather than as an AST piece (e.g. passed it to a proc macro), then we got hygiene bugs.
This PR applies the marks to tokens in addition to the AST pieces thus fixing the issue.
bors [Sun, 8 Nov 2020 13:49:17 +0000 (13:49 +0000)]
Auto merge of #78874 - m-ou-se:rollup-3jp1ijj, r=m-ou-se
Rollup of 19 pull requests
Successful merges:
- #76097 (Stabilize hint::spin_loop)
- #76227 (Stabilize `Poll::is_ready` and `is_pending` as const)
- #78065 (make concurrency helper more pleasant to read)
- #78570 (Remove FIXME comment in print_type_sizes ui test suite)
- #78572 (Use SOCK_CLOEXEC and accept4() on more platforms.)
- #78658 (Add a tool to run `x.py` from any subdirectory)
- #78706 (Fix run-make tests running when LLVM is disabled)
- #78728 (Constantify `UnsafeCell::into_inner` and related)
- #78775 (Bump Rustfmt and RLS)
- #78788 (Correct unsigned equivalent of isize to be usize)
- #78811 (Make some std::io functions `const`)
- #78828 (use single char patterns for split() (clippy::single_char_pattern))
- #78841 (Small cleanup in `TypeFoldable` derive macro)
- #78842 (Honor the rustfmt setting in config.toml)
- #78843 (Less verbose debug logging from inlining integrator)
- #78852 (Convert a bunch of intra-doc links)
- #78860 (rustc_resolve: Use `#![feature(format_args_capture)]`)
- #78861 (typo and formatting)
- #78865 (Don't fire `CONST_ITEM_MUTATION` lint when borrowing a deref)
Mara Bos [Sun, 8 Nov 2020 12:36:33 +0000 (13:36 +0100)]
Rollup merge of #78865 - Aaron1011:fix/const-item-mut-reborrow, r=varkor
Don't fire `CONST_ITEM_MUTATION` lint when borrowing a deref
Fixes #78819
This extends the check for dereferences added in PR #77324
to cover mutable borrows, as well as direct writes. If we're operating
on a dereference of a `const` item, we shouldn't be firing the lint.
Mara Bos [Sun, 8 Nov 2020 12:36:26 +0000 (13:36 +0100)]
Rollup merge of #78843 - tmiasko:inline-trace, r=wesleywiser
Less verbose debug logging from inlining integrator
The inlining integrator produces relatively verbose and uninteresting
logs. Move them from a debug log level to a trace level, so that they
can be easily isolated from others.
Mara Bos [Sun, 8 Nov 2020 12:36:25 +0000 (13:36 +0100)]
Rollup merge of #78842 - shepmaster:bootstrap-rustfmt, r=Mark-Simulacrum
Honor the rustfmt setting in config.toml
Prior to this, setting the rustfmt configuration was ignored:
```
% mkdir example
% cd example
% ../configure --set build.rustfmt=/usr/bin/true
% ../x.py fmt
./x.py fmt is not supported on this channel
failed to run: /Users/shep/Projects/rust/example/build/bootstrap/debug/bootstrap fmt
Build completed unsuccessfully in 0:00:01
```
And after:
```
% ../x.py fmt
Build completed successfully in 0:00:11
```
Mara Bos [Sun, 8 Nov 2020 12:36:18 +0000 (13:36 +0100)]
Rollup merge of #78788 - jhpratt:isize-impl-fix, r=m-ou-se
Correct unsigned equivalent of isize to be usize
See [#74913 (comment)](https://github.com/rust-lang/rust/issues/74913#issuecomment-722334456) for why this matters. Apparently it hasn't been used anywhere else, though CI will tell for sure.
Mara Bos [Sun, 8 Nov 2020 12:36:12 +0000 (13:36 +0100)]
Rollup merge of #78706 - bjorn3:fix_run_make_without_llvm, r=Mark-Simulacrum
Fix run-make tests running when LLVM is disabled
The `--cc`, `--cxx`, `--cflags` and `--ar` flags were only passed to compiletest when `builder.config.llvm_enabled()` returned true. This is preventing me from running the tests on cg_clif.
Mara Bos [Sun, 8 Nov 2020 12:36:09 +0000 (13:36 +0100)]
Rollup merge of #78658 - casey:x, r=Mark-Simulacrum
Add a tool to run `x.py` from any subdirectory
This adds a binary called `x` in `src/tools/x`. All it does is check the current directory and its ancestors for a file called `x.py`, and if it finds one, runs it.
By installing x, you can easily run `x.py` from any subdirectory, and only need to type `x`.
It can be installed with `cargo install --path src/tools/x`
This is a copy of a [binary I've been using myself when working on rust](https://github.com/casey/bootstrap), currently published to crates.io as `bootstrap`.
It could be changed to avoid indirecting through `x.py`, and instead call the bootstrap module directly. However, this seemed like the simplest thing possible, and won't break if the details of how the bootstrap module is invoked change.
Mara Bos [Sun, 8 Nov 2020 12:36:07 +0000 (13:36 +0100)]
Rollup merge of #78572 - de-vri-es:bsd-cloexec, r=m-ou-se
Use SOCK_CLOEXEC and accept4() on more platforms.
This PR enables the use of `SOCK_CLOEXEC` and `accept4` on more platforms.
-----
Android uses the linux kernel, so it should also support it.
DragonflyBSD introduced them in 4.4 (December 2015):
https://www.dragonflybsd.org/release44/
FreeBSD introduced them in 10.0 (January 2014):
https://wiki.freebsd.org/AtomicCloseOnExec
Illumos introduced them in a commit in April 2013, not sure when it was released. It is quite possible that is has always been in Illumos:
https://github.com/illumos/illumos-gate/commit/5dbfd19ad5fcc2b779f40f80fa05c1bd28fd0b4e
https://illumos.org/man/3socket/socket
https://illumos.org/man/3socket/accept4
NetBSD introduced them in 6.0 (Oktober 2012) and 8.0 (July 2018):
https://man.netbsd.org/NetBSD-6.0/socket.2
https://man.netbsd.org/NetBSD-8.0/accept.2
OpenBSD introduced them in 5.7 (May 2015):
https://man.openbsd.org/socket https://man.openbsd.org/accept
Some tests like `multiple_types.rs` are passed even if using `check-pass`. But tests should be agnostic to when the actual layout is computed. The `build-pass` is intentionally used for them. I remove FIXME comments.
Mara Bos [Sun, 8 Nov 2020 12:35:58 +0000 (13:35 +0100)]
Rollup merge of #76227 - CDirkx:const-poll, r=KodrAus
Stabilize `Poll::is_ready` and `is_pending` as const
Insta-stabilize the methods `is_ready` and `is_pending` of `std::task::Poll` as const, in the same way as [PR#76198](https://github.com/rust-lang/rust/pull/76198).
Possible because of the recent stabilization of const control flow.
bors [Sun, 8 Nov 2020 11:27:06 +0000 (11:27 +0000)]
Auto merge of #78410 - lcnr:revert75443, r=nikomatsakis
revert #75443, update mir validator
This PR reverts rust-lang#75443 to fix rust-lang#75992 and instead uses rust-lang#75419 to fix rust-lang#75313.
Adapts rust-lang#75419 to correctly deal with unevaluated constants as otherwise some `feature(const_evaluatable_checked)` tests would ICE.
Note that rust-lang#72793 was also fixed by rust-lang#75443, but as that issue only concerns `feature(type_alias_impl_trait)` I deleted that test case for now and would reopen that issue.
rust-lang#75443 may have also allowed some other code to now successfully compile which would make this revert a breaking change after 2 stable versions, but I hope that this is a purely theoretical concern.
See https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/generator.20upvars/near/214617274 for more reasoning about this.
bors [Sun, 8 Nov 2020 02:21:55 +0000 (02:21 +0000)]
Auto merge of #77729 - petrochenkov:mergetarg, r=Mark-Simulacrum
rustc_target: Move some target options from `Target` to `TargetOptions`
The only reason for `Target` to `TargetOptions` to be separate structures is that options in `TargetOptions` have reasonable defaults and options in `Target` don't.
(Otherwise all the options logically belong to a single `Target` struct.)
This PR moves a number of options with reasonable defaults from `Target` to `TargetOptions`, so they no longer needs to be specified explicitly for majority of the targets.
The move also allows to inherit the options from `rustc_target/src/spec/*_base.rs` files in a nicer way.
I didn't change any specific option values here.
The moved options are `target_c_int_width` (defaults to `"32"`), `target_endian` (defaults to `"little"`), `target_os` (defaults to `"none"`), `target_env` (defaults to `""`), `target_vendor` (defaults to `"unknown"`) and `linker_flavor` (defaults to `LinkerFlavor::Gcc`).
Next steps (in later PRs):
- Find a way to merge `TargetOptions` into `Target`
- If not, always access `TargetOptions` fields through `Deref` making it a part of `Target` at least logically (`session.target.target.options.foo` -> `session.target.target.foo`)
- ~Eliminate `session::config::Config` and use `Target` instead (`session.target.target.foo` -> `session.target.foo`)~ Done in https://github.com/rust-lang/rust/pull/77943.
- Avoid tautologies in option names (`target.target_os` -> `target.os`)
- Resolve _ https://github.com/rust-lang/rust/issues/77730 (rustc_target: The differences between `target_os = "none"` and `target_os = "unknown"`, and `target_vendor = "unknown"` and `target_vendor = ""` are unclear) noticed during implementation of this PR.
Aaron Hill [Sun, 8 Nov 2020 01:11:53 +0000 (20:11 -0500)]
Don't fire `CONST_ITEM_MUTATION` lint when borrowing a deref
Fixes #78819
This extends the check for dereferences added in PR #77324
to cover mutable borrows, as well as direct writes. If we're operating
on a dereference of a `const` item, we shouldn't be firing the lint.
The lang team has decided that for the time being we want to avoid the breakage here (perhaps for a future edition; though almost certainly not the upcoming one), though a future PR may want to add a lint around this case (and perhaps others) which are unlikely to be readable code.
r? `@petrochenkov` to confirm this is the right way to fix #77586.
Tomasz Miąsko [Sat, 7 Nov 2020 00:00:00 +0000 (00:00 +0000)]
Less verbose debug logging from inlining integrator
The inlining integrator produces relatively verbose and uninteresting
logs. Move them from a debug log level to a trace level, so that they
can be easily isolated from others.
Jake Goulding [Sat, 7 Nov 2020 18:26:42 +0000 (13:26 -0500)]
Honor the rustfmt setting in config.toml
Prior to this, setting the rustfmt configuration was ignored:
```
% mkdir example
% cd example
% ../configure --set build.rustfmt=/usr/bin/true
% ../x.py fmt
./x.py fmt is not supported on this channel
failed to run: /Users/shep/Projects/rust/example/build/bootstrap/debug/bootstrap fmt
Build completed unsuccessfully in 0:00:01
```
And after:
```
% ../x.py fmt
Build completed successfully in 0:00:11
```
bors [Sat, 7 Nov 2020 16:54:23 +0000 (16:54 +0000)]
Auto merge of #75199 - Mark-Simulacrum:debug-asserts, r=pietroalbini
Re-enable debug and LLVM assertions
Historically we've disabled these assertions on a number of platforms with the
goal of speeding up CI. Now, though, having migrated to GitHub actions, CI is
already pretty fast, and these debug assertions do bring us some value.
This does leave in some debug assertions that are performance-related: macOS
currently hovers at just under 2 hours.
There are also some other builders which have debug and LLVM assertions
disabled:
llvm-8, PR builder:
In one view, this builder tests our support for older LLVMs. But in reality, a
lot of our tests already disable themselves on older LLVMs, and I think our
general stance is that we really only support the in-tree LLVM. Plus, we really
want CI times on this builder to be really low, as it's run on *every* PR --
that's a lot of CI time.
test-various:
This disables debug asserts still -- as noted in the Dockerfile, we test code
size, and we need debug asserts off for that to work well.
Helps with #59637 -- but doesn't close it, macOS still has asserts off.
Mark Rousskov [Wed, 5 Aug 2020 20:17:18 +0000 (16:17 -0400)]
Re-enable debug and LLVM assertions
Historically we've disabled these assertions on a number of platforms with the
goal of speeding up CI. Now, though, having migrated to GitHub actions, CI is
already pretty fast, and these debug assertions do bring us some value.
This does leave in some debug assertions that are performance-related: macOS
currently hovers at just under 2 hours.
There are also some other builders which have debug and LLVM assertions
disabled:
llvm-8, PR builder:
In one view, this builder tests our support for older LLVMs. But in reality, a
lot of our tests already disable themselves on older LLVMs, and I think our
general stance is that we really only support the in-tree LLVM. Plus, we really
want CI times on this builder to be really low, as it's run on *every* PR --
that's a lot of CI time.
test-various:
This disables debug asserts still -- as noted in the Dockerfile, we test code
size, and we need debug asserts off for that to work well.
bors [Fri, 6 Nov 2020 16:12:13 +0000 (16:12 +0000)]
Auto merge of #78810 - JohnTitor:rollup-8fhtvxu, r=JohnTitor
Rollup of 15 pull requests
Successful merges:
- #74979 (`#![deny(unsafe_op_in_unsafe_fn)]` in sys/hermit)
- #78006 (Use Intra-doc links for std::io::buffered)
- #78167 (Fix unreachable sub-branch detection in or-patterns)
- #78514 (Allow using 1/2/3/4 for `x.py setup` options)
- #78538 (BTreeMap: document a curious assumption in test cases)
- #78559 (Add LLVM upgrades from 7 to 10 to RELEASES.md)
- #78666 (Fix shellcheck error)
- #78705 (Print a summary of which test suite failed)
- #78726 (Add link to rust website)
- #78730 (Expand explanation of reverse_bits)
- #78760 (`deny(invalid_codeblock_attributes)` for rustc_error_codes)
- #78771 (inliner: Copy unevaluated constants only after successful inlining)
- #78794 (rustc_expand: use collect_bang helper instead of manual reimplementation)
- #78795 (The renumber pass is long gone)
- #78798 (Fixing Spelling Typos)
Yuki Okushi [Fri, 6 Nov 2020 16:02:28 +0000 (01:02 +0900)]
Rollup merge of #78795 - est31:node_id_assignment_doc_fix, r=oli-obk
The renumber pass is long gone
Originally, there has been a dedicated pass for renumbering
AST NodeIds to have actual values. This pass had been added by
commit a5ad4c379466519a0bf977864a5cdc50a7ade385.
The comment snippet, added by the original commit, has
survived the times without any change, becoming outdated
at removal of the dedicated pass.
Nowadays, grepping for the next_node_id function will show up
multiple places in the compiler that call it, but the main
rewriting that the comment talks about is still done in the
expansion step, inside an innocious looking visit_id function
that's called during macro invocation collection.
Yuki Okushi [Fri, 6 Nov 2020 16:02:24 +0000 (01:02 +0900)]
Rollup merge of #78771 - tmiasko:inline-consts, r=oli-obk
inliner: Copy unevaluated constants only after successful inlining
Inliner copies the unevaluated constants from the callee body to the
caller at the point where decision to inline is yet to be made. The
constants will be unnecessary if inlining were to fail.
Organize the code moving items from callee to the caller together in one
place to avoid the issue.
Yuki Okushi [Fri, 6 Nov 2020 16:02:16 +0000 (01:02 +0900)]
Rollup merge of #78705 - Mark-Simulacrum:nicer-failure-compiletest, r=jyn514
Print a summary of which test suite failed
Especially on CI, where cross-compiling is common and single builder may end up
with multiple hosts and multiple targets, it can be annoying to scroll back to
the nearest start of test marker. This prints out a summary of the test suite
being run directly in compiletest.
For example, on a mir-opt failure, this would show something like this:
```
failures:
[mir-opt] mir-opt/while-storage.rs
test result: FAILED. 140 passed; 1 failed; 2 ignored; 0 measured; 0 filtered out
Some tests failed in compiletest suite=mir-opt mode=mir-opt host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
```
Yuki Okushi [Fri, 6 Nov 2020 16:02:05 +0000 (01:02 +0900)]
Rollup merge of #78167 - Nadrieril:fix-76836_, r=varkor
Fix unreachable sub-branch detection in or-patterns
The previous implementation was too eager to avoid unnecessary "unreachable pattern" warnings. I feel more confident about this implementation than I felt about the previous one.
Fixes https://github.com/rust-lang/rust/issues/76836.
Yuki Okushi [Fri, 6 Nov 2020 16:01:59 +0000 (01:01 +0900)]
Rollup merge of #74979 - maekawatoshiki:fix, r=Mark-Simulacrum
`#![deny(unsafe_op_in_unsafe_fn)]` in sys/hermit
Partial fix of #73904.
This encloses ``unsafe`` operations in ``unsafe fn`` in ``sys/hermit``.
Some unsafe blocks are not well documented because some system-based functions lack documents.
bors [Fri, 6 Nov 2020 11:31:18 +0000 (11:31 +0000)]
Auto merge of #77351 - jyn514:clippy-sysroot, r=Mark-Simulacrum
Fix `x.py clippy`
I don't think this ever worked.
Fixes https://github.com/rust-lang/rust/issues/77309. `--fix` support is a work in progress, but works for a very small subset of `libtest`.
This works by using the host `cargo-clippy` driver; it does not use `stage0.txt` at all. To mitigate confusion from this, it gives an error if you don't have `rustc +nightly` as the default rustc in `$PATH`. Additionally, it means that bootstrap can't set `RUSTC`; this makes it no longer possible for clippy to detect the sysroot itself. Instead, bootstrap passes the sysroot to cargo.
bors [Fri, 6 Nov 2020 06:59:44 +0000 (06:59 +0000)]
Auto merge of #78267 - richkadel:llvm-coverage-counters-2.0.3r1, r=tmandry
Working expression optimization, and some improvements to branch-level source coverage
This replaces PR #78040 after reorganizing the original commits (by request) into a more logical sequence of major changes.
Most of the work is in the MIR `transform/coverage/` directory (originally, `transform/instrument_coverage.rs`).
Note this PR includes some significant additional debugging capabilities, to help myself and any future developer working on coverage improvements or issues.
In particular, there's a new Graphviz (.dot file) output for the coverage graph (the `BasicCoverageBlock` control flow graph) that provides ways to get some very good insight into the relationships between the MIR, the coverage graph BCBs, coverage spans, and counters. (There are also some cool debugging options, available via environment variable, to alter how some data in the graph appears.)
And the code for this Graphviz view is actually generic... it can be used by any implementation of the Rust `Graph` traits.
Finally (for now), I also now output information from `llvm-cov` that shows the actual counters and spans it found in the coverage map, and their counts (from the `--debug` flag). I found this to be enormously helpful in debugging some coverage issues, so I kept it in the test results as well for additional context.
`@tmandry` `@wesleywiser`
r? `@tmandry`
Here's an example of the new coverage graph:
* Within each `BasicCoverageBlock` (BCB), you can see each `CoverageSpan` and its contributing statements (MIR `Statement`s and/or `Terminator`s)
* Each `CoverageSpan` has a `Counter` or and `Expression`, and `Expression`s show their Add/Subtract operation with nested operations. (This can be changed to show the Counter and Expression IDs instead, or in addition to, the BCB.)
* The terminators of all MIR `BasicBlock`s in the BCB, including one final `Terminator`
* If an "edge counter" is required (because we need to count an edge between blocks, in some cases) the edge's Counter or Expression is shown next to its label. (Not shown in the example below.) (FYI, Edge Counters are converted into a new MIR `BasicBlock` with `Goto`)
Rich Kadel [Fri, 6 Nov 2020 02:21:30 +0000 (18:21 -0800)]
Disable the spanview tests on MacOS for now
And even though CI should now pass for MacOS, the llvm-cov show --debug
flag does not work when developing outside of CI, so I'm disabling it
for MacOS by default.
Rich Kadel [Wed, 4 Nov 2020 05:45:32 +0000 (21:45 -0800)]
rename some tests to avoid exceeding windows path limits
And restored missing error message from llvm-cov show
And since some CI builds disable LLVM assertions (which disables the
--debug option in llvm-cov show), I check to see if LLVM assertions are
disabled, and if so, I don't add --debug and don't check the counter
file diffs.
Rich Kadel [Mon, 2 Nov 2020 23:01:30 +0000 (15:01 -0800)]
fix cross-platform test bugs
More portable way to make python 2/3 portable.
Strip Args line (with hardcoded paths) from debug counters output.
Ignore diff failures from llvm-cov debug output files ("counters"
files), since generic function instantiations will appear in those files
with mangled names. (Sadly, the demangler is apparently not applied to
the debug output.)
Rich Kadel [Thu, 22 Oct 2020 21:30:03 +0000 (14:30 -0700)]
Injecting expressions in place of counters where helpful
Implementing the Graph traits for the BasicCoverageBlock
graph.
optimized replacement of counters with expressions plus new BCB graphviz
* Avoid adding coverage to unreachable blocks.
* Special case for Goto at the end of the body. Make it non-reportable.
Improved debugging and formatting options (from env)
Don't automatically add counters to BCBs without CoverageSpans. They may
still get counters but only if there are dependencies from
other BCBs that have spans, I think.
Make CodeRegions optional for Counters too. It is
possible to inject counters (`llvm.instrprof.increment` intrinsic calls
without corresponding code regions in the coverage map. An expression
can still uses these counter values.
Refactored instrument_coverage.rs -> instrument_coverage/mod.rs, and
then broke up the mod into multiple files.
Compiling with coverage, with the expression optimization, works on
the json5format crate and its dependencies.
est31 [Fri, 6 Nov 2020 02:03:22 +0000 (03:03 +0100)]
The renumber pass is long gone
Originally, there has been a dedicated pass for renumbering
AST NodeIds to have actual values. This pass had been added by
commit a5ad4c379466519a0bf977864a5cdc50a7ade385.
The comment snippet, added by the original commit, has
survived the times without any change, becoming outdated
at removal of the dedicated pass.
Nowadays, grepping for the next_node_id function will show up
multiple places in the compiler that call it, but the main
rewriting that the comment talks about is still done in the
expansion step, inside an innocious looking visit_id function
that's called during macro invocation collection.