Auto merge of #54399 - alexcrichton:fix-bug, r=steveklabnik
std: Check for overflow in `str::repeat`
This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in #48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.
The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!
The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.
Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
- #52813 (Duration div mul extras)
- #53470 (Warn about metadata loader errors)
- #54233 (Remove LLVM 3.9 workaround.)
- #54257 (Switch wasm math symbols to their original names)
- #54258 (Enable fatal warnings for the wasm32 linker)
- #54266 (Update LLVM to fix "bool" arguments on PPC32)
- #54290 (Switch linker for aarch64-pc-windows-msvc from LLD to MSVC)
- #54292 (Suggest array indexing when tuple indexing on an array)
- #54295 (A few cleanups and minor improvements to rustc/traits)
- #54298 (miri: correctly compute expected alignment for field)
- #54333 (Update The Book to latest)
- #54337 (Remove unneeded clone() from tests in librustdoc)
- #54346 (rustc: future-proof error reporting for polymorphic constants in types.)
- #54362 (Pass --batch to gdb)
- #54367 (Add regression test for thread local static mut borrows)
Alex Crichton [Thu, 20 Sep 2018 16:31:14 +0000 (09:31 -0700)]
std: Check for overflow in `str::repeat`
This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in #48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.
The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!
The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.
Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
Rollup merge of #54367 - spastorino:add-thread-local-static-borrow-test, r=pnkfelix
Add regression test for thread local static mut borrows
FIXME(#54366) - We probably shouldn't allow `#[thread_local] static mut` to get a `'static` lifetime, but for now, we should at least test the behavior that `rustc` currently has.
Rollup merge of #54362 - tromey:travis-gdb-batch-mode, r=nikomatsakis
Pass --batch to gdb
In one of my travis builds, I was surprised to find that the gdb
pager was in use and caused travis to time out. Adding `--batch`
to the gdb invocation will disable the pager. Note that the
`-ex q` is retained, to make sure gdb exits with status 0, just in
case `set -e` is in effect somehow.
Rollup merge of #54346 - eddyb:constant-horror, r=nikomatsakis
rustc: future-proof error reporting for polymorphic constants in types.
Currently, we have 3 categories of positions where a constant can be used (`const` and associated `const` can be considered "aliases" for an expression):
* runtime - if the function is polymorphic, we could even just warn and emit a panic
* `static` - always monomorphic, so we can error at definition site
* type-system - **must** *enforce* evaluation success where it was written
That last one is the tricky one, because we can't easily turn *the presence* a type with an erroring const into a runtime panic, and we'd have to do post-monomorphization errors (which we'd rather avoid).
<hr/>
The solution we came up with, as part of the plans for const generics, is to require successful evaluation wherever a constant shows up in a type (currently in array lengths, and values for const parameters in the future), *through* the WF system, which means that in certain situations (e.g. function signatures) we can assume evaluation *will* succeed, and require it of users (e.g. callers) instead (we've been doing this for lifetime bounds, for a long time now, and it's pretty ergonomic).
So once we do sth about #43408, this example *should* work, by propagating the responsability, to callers of `foo::<X>`, of proving `std::mem::size_of::<X>()` succeeds (and those callers can do the same).
```rust
pub fn foo<T>(_: [u8; std::mem::size_of::<T>()]) {}
```
But this one *shouldn't*, as there is nothing in the signature/bounds to indicate it:
```rust
pub fn bar<T>() {
let _: [u8; std::mem::size_of::<T>()];
}
```
<hr/>
I've come across some bit of code in the compiler that ignores const-evaluation errors *even when* they come from a constant in a type, and I've added an ICE *only when* there are no other reported errors (e.g. it's fine to ignore evaluation errors if the constant doesn't even type-check).
Right now, rustdoc pulls all its impl information by scanning a crate's HIR for any items it finds. However, it doesn't recurse into anything other than modules, preventing it from seeing trait impls that may be inside things like functions or consts. Thanks to https://github.com/rust-lang/rust/pull/53002, now these items actually *exist* for rustdoc to see, but they still weren't getting collected for display.
But there was a secret. Whenever we pull in an item from another crate, we don't have any of its impls in the local HIR, so instead we ask the compiler for *everything* and filter out after the fact. This process is only triggered if there's a cross-crate re-export in the crate being documented, which can sometimes leave this info out of the docs. This PR instead moves this collection into an early pass, which occurs immediately after crate cleaning, so that that collection occurs regardless. In addition, by including the HIR's own `trait_impls` in addition to the existing `all_trait_implementations` calls, we can collect all these tricky trait impls without having to scan for them!
Rollup merge of #54295 - ljedrz:cleanups_rustc_traits, r=nikomatsakis
A few cleanups and minor improvements to rustc/traits
It's a little bigger than usual, so bear with me ^^:
- introduce `TyCtxt::all_impls` and use it to avoid inefficiently allocating push loops
- modify `ArgKind::from_expected_ty` to take an `Option<Span>` argument to make it more versatile
- replace `ArgKind::Arg("_".to_owned(), "_".to_owned())` with `ArgKind::empty`
- move early `return`s earlier where possible
- if all branches of a `match` end with the same expression, move it after it
- change a hacky `match` expression to an `if else` chain
- move the `push` out from a push loop closure to reduce the number of allocations
- correct the vector size for `pretty_predicates` (under `specialize`)
- take advantage of the fact that `if else` is an expression
- prefer `cloned()` to `map(|&x| x)` and `map(|x| *x)`
- prefer `vec![x; y.len()]` to `y.map(|_| x).collect()`
- use `unwrap_or_else` instead of `match` where applicable
- use `if let` instead of `match` when only one branch matters
- prefer `to_owned` to `to_string` for string literals
- remove explicit `return`s
- remove superfluous braces
- whitespace fixes
- several other minor improvements
Rollup merge of #54290 - michaelwoerister:aarch64-back-to-msvc, r=alexcrichton
Switch linker for aarch64-pc-windows-msvc from LLD to MSVC
The MSVC linker does not seem to have the same problems with Rust symbols that LLD currently has on Windows (see https://github.com/rust-lang/rust/issues/54190#issuecomment-421288625). This PR makes MSVC the default linker for `aarch64-pc-windows-msvc`.
Rollup merge of #54258 - alexcrichton:lld-fatal-warnings, r=eddyb
Enable fatal warnings for the wasm32 linker
Historically LLD has emitted warnings for various reasons but all the bugs have
since been fixed (yay!) and by enabling fatal warnings we should be able to head
off bugs like #53390 sooner.
Rollup merge of #54257 - alexcrichton:wasm-math-symbols, r=TimNN
Switch wasm math symbols to their original names
The names `Math_*` were given to help undefined symbol messages indicate how to
implement them, but these are all implemented in compiler-rt now so there's no
need to rename them! This change should make it so wasm binaries by default, no
matter the math symbols used, will not have unresolved symbols.
Rollup merge of #53470 - bjorn3:warn_metadata_errors, r=alexcrichton
Warn about metadata loader errors
Output when writing corrupting to libcore.rlib
```
warning: no metadata found: failed to read rlib metadata in '/Users/bjorn/Documents/rust_fork/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps/libcore-857d662d379c5d0c.rlib': File too small to be an archive
Rollup merge of #52813 - newpavlov:duration_mul_div_extras, r=alexcrichton
Duration div mul extras
Successor of #52556.
This PR adds the following `impl`s:
- `impl Mul<Duration> for u32` (to allow `10*SECOND` in addition to `SECOND*10`)
- `impl Mul<f64> for Duration` (to allow `2.5*SECOND` vs `2*SECOND + 500*MILLISECOND`)
- `impl Mul<Duration> for f64`
- `impl MulAssign<f64> for Duration`
- `impl Div<f64> for Duration`
- `impl DivAssign<f64> for Duration`
- `impl Div<Duration> for Duration` (`Output = f64`, can be useful e.g. for `duration/MINUTE`)
`f64` is chosen over `f32` to minimize rounding errors. (52 bits fraction precision vs `Duration`'s ~94 bit)
- clean/inline.rs needs this map to fill in traits when inlining
- fold.rs needs this map to allow passes to fold trait items
- html/render.rs needs this map to seed the Cache.traits map of all
known traits
The first two are the real problem, since `DocFolder` only operates on
`clean::Crate` but `clean/inline.rs` only sees the `DocContext`. The
introduction of early passes means that these two now exist at the same
time, so they need to share ownership of the map. Even better, the use
of `Crate` in a rustc thread pool means that it needs to be Sync, so it
can't use `Lrc<Lock>` to manually activate thread-safety.
`parking_lot` is reused from elsewhere in the tree to allow use of its
`ReentrantMutex`, as the relevant parts of rustdoc are still
single-threaded and this allows for easier use in that context.
Auto merge of #54301 - alexcrichton:update-curl, r=Mark-Simulacrum
Update some `*-sys` dependencies of Cargo/RLS
This is intended to help solve #54206 on nightly where the RLS on MinGW is
having build issues with accidentally building a `curl` library which links to
pthread symbols on Windows (where it should use native mutex locking instead).
The build system for these `*-sys` crates have all been rewritten to be based on
`cc` to bypass native build systems and platform detection to make sure we
configure them correctly.
Alex Crichton [Mon, 17 Sep 2018 16:52:02 +0000 (09:52 -0700)]
Update some `*-sys` dependencies of Cargo/RLS
This is intended to help solve #54206 on nightly where the RLS on MinGW is
having build issues with accidentally building a `curl` library which links to
pthread symbols on Windows (where it should use native mutex locking instead).
The build system for these `*-sys` crates have all been rewritten to be based on
`cc` to bypass native build systems and platform detection to make sure we
configure them correctly.
Auto merge of #54211 - nnethercote:keccak-Liveness-memory, r=nikomatsakis
Split `Liveness::users` into three.
This reduces memory usage on some benchmarks because no space is wasted
for padding. For a `check-clean` build of `keccak` it reduces `max-rss`
by 20%.
r? @nikomatsakis, but I want to do a perf run. Locally, I had these results:
- instructions: slight regression
- max-rss: big win on "Clean" builds
- faults: big win on "Clean" and "Nll" builds
- wall-time: small win on "Clean" and "Nll" builds
So I want to see how a different machine compares.
Auto merge of #54174 - parched:park, r=alexcrichton
Fix `thread` `park`/`unpark` synchronization
Previously the code below would not be guaranteed to exit when the
second unpark took the `return, // already unparked` path because there
was no write to synchronize with a read in `park`.
EDIT: doesn't actually require third thread
```
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread::{current, spawn, park};
while !FLAG.load(Ordering::Relaxed) {
park();
}
}
```
I have some other ideas on how to improve the performance of `park` and `unpark` using fences, avoiding any atomic RMW when the state is already `NOTIFIED`, and also how to avoid calling `notify_one` without the mutex locked. But I need to write some micro benchmarks first, so I'll submit those changes at a later date if they prove to be faster.
Fixes https://github.com/rust-lang/rust/issues/53366 I hope.
Tom Tromey [Wed, 19 Sep 2018 15:46:22 +0000 (09:46 -0600)]
Pass --batch to gdb
In one of my travis builds, I was surprised to find that the gdb
pager was in use and caused travis to time out. Adding `--batch`
to the gdb invocation will disable the pager. Note that the
`-ex q` is retained, to make sure gdb exits with status 0, just in
case `set -e` is in effect somehow.
Auto merge of #54318 - nnethercote:use-HybridBitSet-in-SparseBitMatrix, r=pnkfelix
Use `HybridBitSet` in `SparseBitMatrix`.
This fixes most of the remaining NLL memory regression.
r? @pnkfelix, because you reviewed #54286.
cc @nikomatsakis, because NLL
cc @Mark-Simulacrum, because this removes `array_vec.rs`
cc @lqd, because this massively improves `unic-ucd-name`, and probably other public crates
David Wood [Thu, 6 Sep 2018 13:48:33 +0000 (15:48 +0200)]
De-duplicate moved variable errors.
By introducing a new map that tracks the errors reported and the
`Place`s that spawned those errors against the move out that the error
was referring to, we are able to silence duplicate errors by emitting
only the error which corresponds to the most specific `Place` (that which
other `Place`s which reported errors are prefixes of).
This generally is an improvement, however there is a case -
`liveness-move-in-while` - where the output regresses.
Auto merge of #54319 - GuillaumeGomez:rollup, r=GuillaumeGomez
Rollup of 9 pull requests
Successful merges:
- #53522 (Add doc for impl From for Addr)
- #54097 (rustdoc: Remove namespace for keywords)
- #54205 (Add treat-err-as-bug flag in rustdoc)
- #54225 (Regression test for rust-lang/rust#53675.)
- #54232 (add `-Z dont-buffer-diagnostics`)
- #54273 (Suggest to change numeric literal instead of casting)
- #54299 (Issue 54246)
- #54311 (Remove README with now-out-of-date docs about docs.)
- #54313 (OsStr: Document that it's not NUL terminated)
Rollup merge of #54313 - cgwalters:osstr-ref-cstr, r=joshtriplett
OsStr: Document that it's not NUL terminated
I somehow got confused into thinking this was the case, but
it's definitely not. Let's help the common case of people who
have an `OsStr` and need to call e.g. Unix APIs.
Rollup merge of #54299 - snaedis:issue-54246, r=varkor
Issue 54246
I added the option of providing a help message for deprecated features, that takes precedence over the default `help: remove this attribute` message, along with messages for the features that mention replacements in the reason for deprecation.
Rollup merge of #54232 - pnkfelix:add-way-to-disable-diagnostic-buffering, r=nikomatsakis
add `-Z dont-buffer-diagnostics`
Add `-Z dont-buffer-diagnostics`, a way to force NLL to immediately its diagnostics.
This is mainly intended for developers who want to see the error in its original context in the control flow. Two uses cases for that are:
1. `-Z treat-err-as-bug` (which then allows extraction of a stack-trace to the origin of the error)
2. RUST_LOG=... rustc, in which case it is often useful to see the logging statements that occurred immediately prior to the point where the diagnostic was signalled.
Rollup merge of #54225 - pnkfelix:issue-53675-add-test-called-panic, r=petrochenkov
Regression test for rust-lang/rust#53675.
(Includes a couple variations on the theme. I confirmed that the ones
in `in_expression_position` and `what_if_we_use_panic_directly_in_expr`
both failed back on "rustc 1.30.0-nightly (0f063aef6 2018-09-03)".)
Use `HybridBitSet` for rows within `SparseBitMatrix`.
This requires adding a few extra methods to `HybridBitSet`. (These are
tested in a new unit test.)
This commit reduces the `max-rss` for `nll-check` builds of `html5ever`
by 46%, `ucd` by 45%, `clap-rs` by 23%, `inflate` by 14%. And the
results for the `unic-ucd-name` crate are even more impressive: a 21%
reduction in instructions, a 60% reduction in wall-time, a 96%
reduction in `max-rss`, and a 97% reduction in faults!
`SparseBitSet` is the only remaining user of `ArrayVec`. This commit
switches it to using `SmallVec`, and removes `array_vec.rs`.
Why the switch? Although `SparseBitSet` is size-limited and doesn't need
the ability to spill to the heap, `SmallVec` has many more features than
`ArrayVec`. In particular, it's now possible to keep `SparseBitSet`'s
elements in sorted order, which gives in-order iteration, which is a
requirement for the next commit.
Auto merge of #54286 - nnethercote:BitSet, r=pnkfelix
Merge `bitvec.rs` and `indexed_set.rs`
Because it's not good to have two separate implementations. Also, I will combine the best parts of each to improve NLL memory usage on some benchmarks significantly.
Colin Walters [Tue, 18 Sep 2018 01:10:36 +0000 (21:10 -0400)]
OsStr: Document that it's not NUL terminated
I somehow got confused into thinking this was the case, but
it's definitely not. Let's help the common case of people who
have an `OsStr` and need to call e.g. Unix APIs.
`BitwiseOperator` is an unnecessarily low-level thing. This commit
replaces it with `BitSetOperator`, which works on `BitSet`s instead of
words. Within `bit_set.rs`, the commit eliminates `Intersect`, `Union`,
and `Subtract` by instead passing a function to `bitwise()`.
Merge indexed_set.rs into bitvec.rs, and rename it bit_set.rs.
Currently we have two files implementing bitsets (and 2D bit matrices).
This commit combines them into one, taking the best features from each.
This involves renaming a lot of things. The high level changes are as
follows.
- bitvec.rs --> bit_set.rs
- indexed_set.rs --> (removed)
- BitArray + IdxSet --> BitSet (merged, see below)
- BitVector --> GrowableBitSet
- {,Sparse,Hybrid}IdxSet --> {,Sparse,Hybrid}BitSet
- BitMatrix --> BitMatrix
- SparseBitMatrix --> SparseBitMatrix
The changes within the bitset types themselves are as follows.
```
OLD OLD NEW
BitArray<C> IdxSet<T> BitSet<T>
-------- ------ ------
grow - grow
new - (remove)
new_empty new_empty new_empty
new_filled new_filled new_filled
- to_hybrid to_hybrid
clear clear clear
set_up_to set_up_to set_up_to
clear_above - clear_above
count - count
contains(T) contains(&T) contains(T)
contains_all - superset
is_empty - is_empty
insert(T) add(&T) insert(T)
insert_all - insert_all()
remove(T) remove(&T) remove(T)
words words words
words_mut words_mut words_mut
- overwrite overwrite
merge union union
- subtract subtract
- intersect intersect
iter iter iter
```
In general, when choosing names I went with:
- names that are more obvious (e.g. `BitSet` over `IdxSet`).
- names that are more like the Rust libraries (e.g. `T` over `C`,
`insert` over `add`);
- names that are more set-like (e.g. `union` over `merge`, `superset`
over `contains_all`, `domain_size` over `num_bits`).
Also, using `T` for index arguments seems more sensible than `&T` --
even though the latter is standard in Rust collection types -- because
indices are always copyable. It also results in fewer `&` and `*`
sigils in practice.
Auto merge of #54277 - petrochenkov:afterder, r=alexcrichton
Temporarily prohibit proc macro attributes placed after derives
... and also proc macro attributes used together with `#[test]`/`#[bench]`.
Addresses item 6 from https://github.com/rust-lang/rust/pull/50911#issuecomment-411605393.
The end goal is straightforward predictable left-to-right expansion order for attributes.
Right now derives are expanded last regardless of their relative ordering with macro attributes and right now it's simpler to temporarily prohibit macro attributes placed after derives than changing the expansion order.
I'm not sure whether the new beta is already released or not, but if it's released, then this patch needs to be backported, so the solution needs to be minimal.
How to fix broken code (derives):
- Move macro attributes above derives. This won't change expansion order, they are expanded before derives anyway.
Using attribute macros on same items with `#[test]` and `#[bench]` is prohibited for similar expansion order reasons, but this one is going to be reverted much sooner than restrictions on derives.
How to fix broken code (test/bench):
- Enable `#![feature(plugin)]` (don't ask why).
Add -Z dont-buffer-diagnostics, a way to force NLL to immediately emit its diagnostics.
This is mainly intended for `rustc` developers who want to see a
diagnostic in its original context in the control flow. Two uses
cases for that are:
* `-Z treat-err-as-bug` which then allows extraction of a stack-trace to the origin of the error
(a case that is so important that we make that flag imply this one, effectively).
* `RUST_LOG=... rustc`, in which case it is often useful to see the logging statements that
occurred immediately prior to the point where the diagnostic was signalled.
Drive-by: Added some documentation pointing future devs at
HandlerFlags, and documented the fields of `HandlerFlags` itself.