bors [Sun, 18 Oct 2020 16:10:00 +0000 (16:10 +0000)]
Auto merge of #77306 - lcnr:inline-ok, r=eddyb
normalize substs while inlining
fixes #68347 or more precisely, this fixes the same ICE in rust analyser as veloren is pinned to a specific nightly
and had an error with the current one.
I didn't look into creating an MVCE here as that seems fairly annoying, will spend a few minutes doing so rn. (failed)
bors [Sun, 18 Oct 2020 11:19:14 +0000 (11:19 +0000)]
Auto merge of #78058 - bugadani:arena2, r=lcnr
Make sure arenas don't allocate bigger than HUGE_PAGE
Right now, arenas allocate based on the size of the last chunk. It is possible for a `grow` call to allocate a chunk that is not a multiple of `PAGE`, and this size is doubled for each subsequent allocation. This means, instead of `HUGE_PAGE`, the biggest page possible is actually unknown.
This change fixes this, and also removes an unnecessary checked multiplication. It is still possible to allocate bigger than `HUGE_PAGE` pages, but this will only happen as many times as absolutely necessary.
bors [Sun, 18 Oct 2020 05:36:23 +0000 (05:36 +0000)]
Auto merge of #78047 - khyperia:set_span_mut, r=bjorn3
Make set_span take mut self
This was a mistake in https://github.com/rust-lang/rust/pull/77614
It's not a _huge_ deal, because backends can always implement this with interior mutability, but it's nice to avoid interior mutability when possible. For context, the `set_source_location` method, called alongside `set_span`, also takes `&mut self`.
bors [Sun, 18 Oct 2020 02:28:21 +0000 (02:28 +0000)]
Auto merge of #74480 - yoshuawuyts:hardware_threads, r=dtolnay
Add std::thread::available_concurrency
This PR adds a counterpart to [C++'s `std::thread::hardware_concurrency`](https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency) to Rust, tracking issue https://github.com/rust-lang/rust/issues/74479.
cc/ `@rust-lang/libs`
## Motivation
Being able to know how many hardware threads a platform supports is a core part of building multi-threaded code. In C++ 11 this has become available through the [`std::thread::hardware_concurrency`](https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency) API. Currently in Rust most of the ecosystem depends on the [`num_cpus` crate](https://docs.rs/num_cpus/1.13.0/num_cpus/) ([no.35 in top 500 crates](https://docs.google.com/spreadsheets/d/1wwahRMHG3buvnfHjmPQFU4Kyfq15oTwbfsuZpwHUKc4/edit#gid=1253069234)) to provide this functionality. This PR proposes an API to provide access to the number of hardware threads available on a given platform.
__edit (2020-07-24):__ The purpose of this PR is to provide a hint for how many threads to spawn to saturate the processor. There's value in introducing APIs for NUMA and Windows processor groups, but those are intentionally out of scope for this PR. See: https://github.com/rust-lang/rust/pull/74480#issuecomment-662116186.
## Naming
Discussing the naming of the API on Zulip surfaced two options:
Both options seemed acceptable, but overall people seem to gravitate the most towards `hardware_threads`. Additionally `@jonas-schievink` pointed out that the "hardware threads" terminology is well-established and is used in among other the [RISC-V specification](https://riscv.org/specifications/isa-spec-pdf/) (page 20):
> A component is termed a core if it contains an independent instruction fetch unit. A RISC-V-compatible core might support multiple RISC-V-compatible __hardware threads__, or harts, through multithreading.
It's also worth noting that [the original paper introducing C++'s `std::thread` submodule](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2320.html) unfortunately doesn't feature any discussion on the naming of `hardware_concurrency`, so we can't use that to help inform our decision here.
## Return type
An important consideration `@joshtriplett` brought up is that we don't want to default to `1` for platforms where the number of available threads cannot be retrieved. Instead we want to inform the users of the fact that we don't know and allow them to handle that case. Which is why this PR uses `Option<NonZeroUsize>` as its return type, where `None` is returned on platforms where we don't know the number of hardware threads available.
The reasoning for `NonZeroUsize` vs `usize` is that if the number of threads for a platform are known, they'll always be at least 1. As evidenced by the example the `NonZero*` family of APIs may currently not be the most ergonomic to use, but improving the ergonomics of them is something that I think we can address separately.
## Implementation
`@Mark-Simulacrum` pointed out that most of the code we wanted to expose here was already available under `libtest`. So this PR mostly moves the internal code of libtest into a public API.
bors [Sun, 18 Oct 2020 00:19:24 +0000 (00:19 +0000)]
Auto merge of #77956 - JohnTitor:more-tests, r=nagisa
Add some more regression tests
This is another round of #77741. Tested with `debug-assertions=true` and it passed on my local.
Closes #70877
Closes #70944
Closes #71659
Closes #74816
Closes #75707
Closes #75983
(Skipped #63355 because I'm not sure about the error.)
bors [Sat, 17 Oct 2020 19:39:26 +0000 (19:39 +0000)]
Auto merge of #78060 - JohnTitor:rollup-uou8vyu, r=JohnTitor
Rollup of 7 pull requests
Successful merges:
- #75802 (resolve: Do not put nonexistent crate `meta` into prelude)
- #76607 (Modify executable checking to be more universal)
- #77851 (BTreeMap: refactor Entry out of map.rs into its own file)
- #78043 (Fix grammar in note for orphan-rule error [E0210])
- #78048 (Suggest correct place to add `self` parameter when inside closure)
- #78050 (Small CSS cleanup)
- #78059 (Set `MDBOOK_OUTPUT__HTML__INPUT_404` on linkchecker)
Yuki Okushi [Sat, 17 Oct 2020 19:11:11 +0000 (04:11 +0900)]
Rollup merge of #78048 - blyxxyz:e0424-improve-self-placement, r=lcnr
Suggest correct place to add `self` parameter when inside closure
It would incorrectly suggest adding it as a parameter to the closure instead of the containing function.
[For example](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1936bcd1e5f981573386e0cee985c3c0):
```
help: add a `self` receiver parameter to make the associated `fn` a method
|
5 | let _ = || self&self;
| ^^^^^
```
`DiagnosticMetadata.current_function` is only used for these messages so tweaking its behavior should be ok.
Yuki Okushi [Sat, 17 Oct 2020 19:11:03 +0000 (04:11 +0900)]
Rollup merge of #75802 - petrochenkov:nometa, r=nikomatsakis
resolve: Do not put nonexistent crate `meta` into prelude
Before the 2018 edition release there was some vague suggestion about adding a crate named `meta` to the standard distribution.
On this basis the name `meta` was "partially reserved" by putting `meta` into extern prelude (this means importing something named `meta` will result in an ambiguity error, for example).
This only caused confusion so far, and two years later there are no specific plans to add such crate.
If some standard crate (named `meta` or not) is added in the future, then cargo will hopefully already have ability to put it into extern prelude explicitly through `Cargo.toml`.
Otherwise, it could be added to extern prelude by the compiler at edition boundary.
bors [Sat, 17 Oct 2020 16:57:21 +0000 (16:57 +0000)]
Auto merge of #77373 - jonas-schievink:rm-rf-copy-prop, r=oli-obk
Remove the old copy propagation pass
This pass was added a long time ago, and has not really seen much improvement since (apart from some great work in https://github.com/rust-lang/rust/pull/76569 that unfortunately ran into preexisting soundness issues). It is slow and unsound, and we now have a destination propagation pass that performs a related optimization and could be extended.
bors [Sat, 17 Oct 2020 06:16:00 +0000 (06:16 +0000)]
Auto merge of #77455 - asm89:faster-spawn, r=kennytm
Use posix_spawn() on unix if program is a path
Previously `Command::spawn` would fall back to the non-posix_spawn based
implementation if the `PATH` environment variable was possibly changed.
On systems with a modern (g)libc `posix_spawn()` can be significantly
faster. If program is a path itself the `PATH` environment variable is
not used for the lookup and it should be safe to use the
`posix_spawnp()` method. [1]
We found this, because we have a cli application that effectively runs a
lot of subprocesses. It would sometimes noticeably hang while printing
output. Profiling showed that the process was spending the majority of
time in the kernel's `copy_page_range` function while spawning
subprocesses. During this time the process is completely blocked from
running, explaining why users were reporting the cli app hanging.
Through this we discovered that `std::process::Command` has a fast and
slow path for process execution. The fast path is backed by
`posix_spawnp()` and the slow path by fork/exec syscalls being called
explicitly. Using fork for process creation is supposed to be fast, but
it slows down as your process uses more memory. It's not because the
kernel copies the actual memory from the parent, but it does need to
copy the references to it (see `copy_page_range` above!). We ended up
using the slow path, because the command spawn implementation in falls
back to the slow path if it suspects the PATH environment variable was
changed.
Here is a smallish program demonstrating the slowdown before this code
change:
```
use std::process::Command;
use std::time::Instant;
fn main() {
let mut args = std::env::args().skip(1);
if let Some(size) = args.next() {
// Allocate some memory
let _xs: Vec<_> = std::iter::repeat(0)
.take(size.parse().expect("valid number"))
.collect();
let mut command = Command::new("/bin/sh");
command
.arg("-c")
.arg("echo hello");
if args.next().is_some() {
println!("Overriding PATH");
command.env("PATH", std::env::var("PATH").expect("PATH env var"));
}
let now = Instant::now();
let child = command
.spawn()
.expect("failed to execute process");
println!("Spawn took: {:?}", now.elapsed());
let output = child.wait_with_output().expect("failed to wait on process");
println!("Output: {:?}", output);
} else {
eprintln!("Usage: prog [size]");
std::process::exit(1);
}
()
}
```
Running it and passing different amounts of elements to use to allocate
memory shows that the time taken for `spawn()` can differ quite
significantly. In latter case the `posix_spawnp()` implementation is 30x
faster:
Dylan DPC [Sat, 17 Oct 2020 01:27:18 +0000 (03:27 +0200)]
Rollup merge of #77932 - ssomers:btree_cleanup_gdb, r=Mark-Simulacrum
BTreeMap: improve gdb introspection of BTreeMap with ZST keys or values
I accidentally pushed an earlier revision in #77788: it changes the index of tuples for BTreeSet from ""[{}]".format(i) to "key{}".format(i). Which doesn't seem to make the slightest difference on my linux box nor on CI. In fact, gdb doesn't make any distinction between "key{}" and "val{}" for a BTreeMap either, leading to confusing output if you test more. But easy to improve.
Dylan DPC [Sat, 17 Oct 2020 01:27:12 +0000 (03:27 +0200)]
Rollup merge of #76199 - Mark-Simulacrum:void-zero, r=nikomatsakis
Permit uninhabited enums to cast into ints
This essentially reverts part of #6204; it is unclear why that [commit](https://github.com/rust-lang/rust/pull/6204/commits/c0f587de34f30b060df8a88c4068740e587b9340) was introduced, and I suspect no one remembers.
The changed code was only called from casting checks and appears to not affect any callers of that code (other than permitting this one case).
bors [Fri, 16 Oct 2020 22:53:50 +0000 (22:53 +0000)]
Auto merge of #77997 - fusion-engineering-forks:to-string-no-shrink, r=joshtriplett
Remove shrink_to_fit from default ToString::to_string implementation.
As suggested by `@scottmcm` on Zulip. shrink_to_fit() seems like the wrong thing to do here in most use cases of to_string(). Would be intereseting to see if it makes any difference in a timer run.
bors [Fri, 16 Oct 2020 20:43:27 +0000 (20:43 +0000)]
Auto merge of #78028 - JohnTitor:rollup-jt3hikb, r=JohnTitor
Rollup of 10 pull requests
Successful merges:
- #75209 (Suggest imports of unresolved macros)
- #77547 (stabilize union with 'ManuallyDrop' fields and 'impl Drop for Union')
- #77827 (Don't link to nightly primitives on stable channel)
- #77855 (resolve: further improvements to "try using the enum's variant" diagnostic)
- #77900 (Use fdatasync for File::sync_data on more OSes)
- #77925 (Suggest minimal subset features in `incomplete_features` lint)
- #77971 (Deny broken intra-doc links in linkchecker)
- #77991 (Bump backtrace-rs)
- #77992 (instrument-coverage: try our best to not ICE)
- #78013 (Fix sidebar scroll on mobile devices)
Yuki Okushi [Fri, 16 Oct 2020 20:36:54 +0000 (05:36 +0900)]
Rollup merge of #78013 - GuillaumeGomez:fix-sidebar-scroll-mobile-devices, r=jyn514
Fix sidebar scroll on mobile devices
Fixes #77942.
The issue was coming from the appearance/disappearance of the "wrapper" on the mobile devices web browsers, which triggers the "resize" event, calling the `hideSidebar` function is the JS code.
Yuki Okushi [Fri, 16 Oct 2020 20:36:52 +0000 (05:36 +0900)]
Rollup merge of #77992 - nagisa:thaw-coverage-instrumentation, r=wesleywiser
instrument-coverage: try our best to not ICE
instrument-coverage was ICEing for me on some code, in particular code
that had devirtualized paths from standard library. Instrument coverage
probably has no bussiness dictating which paths are valid and which
aren't so just feed it everything and whatever and let tooling deal with
other stuff.
For example, with this commit we can generate coverage hitpoints for
these interesting paths:
* `/rustc/.../library/core/lib.rs` – non-devirtualized path for libcore
* `/home/.../src/library/core/lib.rs` – devirtualized version of above
* `<inline asm>`, `<anon>` and many similar synthetic paths
Even if those paths somehow get to the instrumentation pass, I'd much
rather get hits for these weird paths and hope some of them work (as
would be the case for devirtualized path to libcore), rather than have
compilation fail entirely.
Yuki Okushi [Fri, 16 Oct 2020 20:36:47 +0000 (05:36 +0900)]
Rollup merge of #77925 - JohnTitor:sugg-min-features, r=davidtwco,oli-obk
Suggest minimal subset features in `incomplete_features` lint
This tells users that we have a minimal subset feature of it and they can fix the lint warning without allowing it.
The wording improvement is helpful :)
Yuki Okushi [Fri, 16 Oct 2020 20:36:45 +0000 (05:36 +0900)]
Rollup merge of #77900 - Thomasdezeeuw:fdatasync, r=dtolnay
Use fdatasync for File::sync_data on more OSes
Add support for the following OSes:
* Android
* FreeBSD: https://www.freebsd.org/cgi/man.cgi?query=fdatasync&sektion=2
* OpenBSD: https://man.openbsd.org/OpenBSD-5.8/fsync.2
* NetBSD: https://man.netbsd.org/fdatasync.2
* illumos: https://illumos.org/man/3c/fdatasync
Yuki Okushi [Fri, 16 Oct 2020 20:36:43 +0000 (05:36 +0900)]
Rollup merge of #77855 - davidtwco:pr-77341-follow-up-non-constructable-variants, r=estebank
resolve: further improvements to "try using the enum's variant" diagnostic
Follow-up on https://github.com/rust-lang/rust/pull/77341#issuecomment-702738281.
This PR improves the diagnostic modified in #77341 to suggest not only those variants which do not have fields, but those with fields (by suggesting with placeholders). In addition, the wording of the tuple-variant-only case is improved slightly.
I've not made further changes to the tuple-variant-only case (e.g. to only suggest variants with the correct number of fields) because I don't think I have enough information to do so reliably (e.g. in the case where there is an attempt to construct a tuple variant, I have no information on how many fields were provided; and in the case of pattern matching, I only have a slice of spans and would need to check for things like `..` in those spans, which doesn't seem worth it).
Yuki Okushi [Fri, 16 Oct 2020 20:36:38 +0000 (05:36 +0900)]
Rollup merge of #77547 - RalfJung:stable-union-drop, r=matthewjasper
stabilize union with 'ManuallyDrop' fields and 'impl Drop for Union'
As [discussed by @SimonSapin and @withoutboats](https://github.com/rust-lang/rust/issues/55149#issuecomment-634692020), this PR proposes to stabilize parts of the `untagged_union` feature gate:
* It will be possible to have a union with field type `ManuallyDrop<T>` for any `T`.
* While at it I propose we also stabilize `impl Drop for Union`; to my knowledge, there are no open concerns around this feature.
In the RFC discussion, we also talked about allowing `&mut T` as another non-`Copy` non-dropping type, but that felt to me like an overly specific exception so I figured we'd wait if there is actually any use for such a special case.
Some things remain unstable and still require the `untagged_union` feature gate:
* Union with fields that do not drop, are not `Copy`, and are not `ManuallyDrop<_>`. The reason to not stabilize this is to avoid semver concerns around libraries adding `Drop` implementations later. (This is already not fully semver compatible as, to my knowledge, the borrow checker will exploit the non-dropping nature of any type, but it seems prudent to avoid further increasing the amount of trouble adding an `impl Drop` can cause.)
Due to this, quite a few tests still need the `untagged_union` feature, but I think the ones where I could remove the feature flag provide good test coverage for the stable part.
instrument-coverage was ICEing for me on some code, in particular code
that had devirtualized paths from standard library. Instrument coverage
probably has no bussiness dictating which paths are valid and which
aren't so just feed it everything and whatever and let tooling deal with
other stuff.
For example, with this commit we can generate coverage hitpoints for
these interesting paths:
* `/rustc/.../library/core/lib.rs` – non-devirtualized path for libcore
* `/home/.../src/library/core/lib.rs` – devirtualized version of above
* `<inline asm>`, `<anon>` and many similar synthetic paths
Even if those paths somehow get to the instrumentation pass, I'd much
rather get hits for these weird paths and hope some of them work (as
would be the case for devirtualized path to libcore), rather than have
compilation fail entirely.
bors [Fri, 16 Oct 2020 14:06:37 +0000 (14:06 +0000)]
Auto merge of #77972 - Mark-Simulacrum:side-effect-loop, r=nagisa
Prevent miscompilation in trivial loop {}
Ideally, we would want to handle a broader set of cases to fully fix the
underlying bug here. That is currently relatively expensive at compile and
runtime, so we don't do that for now.
Performance results indicate this is not a major regression, if at all, so it should be safe to land.