bors [Tue, 24 May 2022 15:17:33 +0000 (15:17 +0000)]
Auto merge of #2138 - JakobDegen:call-fallout, r=RalfJung
Adjust Miri to also require return places everywhere
This is the miri side of rust-lang/rust#96098 . It'll still need a bump to rust-version once the rust PR is merged, but the test suite passes against my local build of rustc.
bors [Mon, 23 May 2022 18:02:34 +0000 (18:02 +0000)]
Auto merge of #2152 - RalfJung:rustup, r=RalfJung
rustup
On macOS the timing tests [failed again](https://github.com/rust-lang/miri/runs/6557930217?check_suite_focus=true), so let's give them even more slack.
bors [Mon, 23 May 2022 10:39:51 +0000 (10:39 +0000)]
Auto merge of #2150 - RalfJung:rustup, r=RalfJung
Rustup
In particular, this contains https://github.com/rust-lang/rust/pull/97219. Miri support for actually erroring on misuse of `ptr::invalid` will happen later.
bors [Mon, 23 May 2022 09:25:16 +0000 (09:25 +0000)]
Auto merge of #2147 - RalfJung:readme, r=RalfJung
split flag section into common and advanced flags
As discussed with `@oli-obk` . However I was not always sure which flags to put where, so if you think some flags should be in the other category please let me know. :)
bors [Mon, 23 May 2022 07:24:33 +0000 (07:24 +0000)]
Auto merge of #2059 - carbotaniuman:master, r=RalfJung
Initial work on Miri permissive-exposed-provenance
Miri portions of the changes for portions of a permissive ptr-to-int model for Miri. This is more restrictive than what we currently have so it will probably need a flag once I figure out how to hook that up.
> This implements a form of permissive exposed-address provenance, wherein the only way to expose the address is with a cast to usize (ideally expose_addr). This is more restrictive than C in that stuff like reading the representation bytes (via unions, type-punning, transmute) does not expose the address, only expose_addr. This is less restrictive than C in that a pointer casted from an integer has union provenance of all exposed pointers, not any udi stuff.
There's a few TODOs here, namely related to `fn memory_read` and friends. We pass it the maybe/unreified provenance before `ptr_get_alloc` reifies it into a concrete one, so it doesn't have the `AllocId` (or the SB tag, but that's getting ahead of ourselves). One way this could be fixed is changing `ptr_get_alloc` and (`ptr_try_get_alloc_id` on the rustc side) to return a pointer with the tag fixed up. We could also take in different arguments, but I'm not sure what works best.
The other TODOs here are how permissive this model could be. This currently does not enforce that a ptr-to-int cast happens before the corresponding int-to-ptr (colloquial meaning of happens before, not atomic meaning). Example:
```
let ptr = 0x2000 as *const i32;
let a: i32 = 5;
let a_ptr = &a as *const i32;
// value is 0x2000;
a_ptr as usize;
println!("{}", unsafe { *ptr }); // this is valid
```
We also allow the resulting pointer to dereference different non-contiguous allocations (the "not any udi stuff" mentioned above), which I'm not sure if is allowed by LLVM.
This is the Miri side of https://github.com/rust-lang/rust/pull/95826.
bors [Mon, 23 May 2022 06:46:46 +0000 (06:46 +0000)]
Auto merge of #2139 - saethlin:lazy-current-span, r=RalfJung
Factor current-span logic into a caching handle
After https://github.com/rust-lang/miri/pull/2030 and while working on https://github.com/rust-lang/miri/pull/1935 it became quite clear that we need to do some caching here, because some retag operations generate many calls to `log_invalidation`, and would thus search the current thread's stack _many_ times for a local crate. This caching fixes that. This handle type also has the nice benefit of tucking away all the `ThreadManager` + `CrateNum` logic.
bors [Sun, 22 May 2022 20:00:17 +0000 (20:00 +0000)]
Auto merge of #2144 - matthiaskrgr:stderrfix, r=RalfJung
mute_stdout_stderr: mute stderr instead of stdin
should fix #2143
note: this is entirely untested, I was getting tons of errors on `cargo test` because of some missing setup.
I hope that CI can tell me if this works or not :see_no_evil:
bors [Sun, 22 May 2022 16:17:17 +0000 (16:17 +0000)]
Auto merge of #2141 - saethlin:early-diagnostics-ice, r=RalfJung
Adjust diagnostics assertion so we don't ICE in setup
Fixes https://github.com/rust-lang/miri/issues/2076 just by handling diagnostics produced during setup. The tracking notes don't have any spans but it's better than an ICE.
It looks like we leak allocations 1..20, and allocations 13..19 don't have any creation notes, and 14 only has a `FreedAlloc` alloc tracking diagnostic.
I've tried for hours now to come up with a test case for this ICE with no luck. I suspect there's something about the way the data race detection works under these conditions that I just don't understand :weary:.
But I tried this change out on a handful of crates and I don't see any more ICEs of this form. For whatever reason it seems like `bastion==0.4.5` is a good way to run into this, with the flags
```
MIRIFLAGS="-Zmiri-tag-raw-pointers -Zmiri-panic-on-unsupported -Zmiri-disable-isolation" cargo +miri miri test --no-fail-fast --doc
```
I think all the cases I've run into with this involve both `-Zmiri-panic-on-unsupported` and `-Zmiri-tag-raw-pointers`, so it could be that the combination of an unexpected panic and a machine halt is required.
bors [Sun, 22 May 2022 05:59:39 +0000 (05:59 +0000)]
Auto merge of #2140 - V0ldek:page_size, r=RalfJung
Update GetSystemInfo to work with `page_size` (#2136)
- Change logic in GetSystemInfo shim to take into account the two possible layouts of `SYSTEM_INFO`, the first-field-is-union used by [winapi::um::sysinfoapi::SYSTEM_INFO](https://docs.rs/winapi/latest/winapi/um/sysinfoapi/struct.SYSTEM_INFO.html), and first-two-fields-are-inlined-union used by [num_cpus](https://github.com/seanmonstar/num_cpus/blob/5f1b03332000b4c4274b5bd35fac516049ff1c6b/src/lib.rs#L206).
- Fill out the `dwPageSize` field with the `PAGE_SIZE` constant of `4096`.
bors [Sat, 21 May 2022 16:15:49 +0000 (16:15 +0000)]
Auto merge of #2137 - RalfJung:targets, r=oli-obk
explain which targets we support to what extent
This is basically documenting my policy so far: Linux is the target I know best and can spend most time on. Apple is so close to Linux that it can basically ride along without much extra work. I don't have a lot of time to spend on our Windows support.
Of course, if people commit to contributing and maintaining support for a target, we can promise more than what is documented here. :) But this is what I am willing to promise.
Also reduce the amount of work we do on the Windows test runner, since that one currently takes 10min longer than the other two.
bors [Sat, 14 May 2022 19:27:04 +0000 (19:27 +0000)]
Auto merge of #2030 - saethlin:track-alloc-history, r=oli-obk
Print spans where tags are created and invalidated
5225225 called this "automatic tag tracking" and I think that may be a reasonable description, but I would like to kill tag tracking as a primary use of Miri if possible. Tag tracking isn't always possible; for example if the UB is only detected with isolation off and the failing tag is made unstable by removing isolation. (also it's bad UX to run the tool twice)
This is just one of the things we can do with https://github.com/rust-lang/miri/pull/2024
The memory usage of this is _shockingly_ low, I think because the memory usage of Miri is driven by allocations where each byte ends up with its own very large stack. The memory usage in this change is linear with the number of tags, not tags * bytes. If memory usage gets out of control we can cap the number of events we save per allocation, from experience we tend to only use the most recent few in diagnostics but of course there's no guarantee of that so if we can manage to keep everything that would be best.
In many cases now I can tell exactly what these codebases are doing wrong just from the new outputs here, which I think is extremely cool.
New helps generated with plain old `cargo miri test` on `rust-argon2` v1.0.0:
```
test argon2::tests::single_thread_verification_multi_lane_hash ... error: Undefined Behavior: trying to reborrow <1485898> for Unique permission at alloc110523[0x0], but that tag does not exist in the borrow stack for this location
--> /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/mem/manually_drop.rs:89:9
|
89 | slot.value
| ^^^^^^^^^^
| |
| trying to reborrow <1485898> for Unique permission at alloc110523[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of a reborrow at alloc110523[0x0..0x20]
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <1485898> was created by a retag at offsets [0x0..0x20]
--> src/memory.rs:42:13
|
42 | vec.push(unsafe { &mut (*ptr) });
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: <1485898> was later invalidated at offsets [0x0..0x20]
--> src/memory.rs:42:31
|
42 | vec.push(unsafe { &mut (*ptr) });
| ^^^^^^^^^^^
```
And with `-Zmiri-tag-raw-pointers` on `slab` v0.4.5
```
error: Undefined Behavior: trying to reborrow <2915> for Unique permission at alloc1418[0x0], but that tag does not exist in the borrow stack for this location
--> /tmp/slab-0.4.5/src/lib.rs:835:16
|
835 | match (&mut *ptr1, &mut *ptr2) {
| ^^^^^^^^^^
| |
| trying to reborrow <2915> for Unique permission at alloc1418[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of a reborrow at alloc1418[0x0..0x10]
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <2915> was created by a retag at offsets [0x0..0x10]
--> /tmp/slab-0.4.5/src/lib.rs:833:20
|
833 | let ptr1 = self.entries.get_unchecked_mut(key1) as *mut Entry<T>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: <2915> was later invalidated at offsets [0x0..0x20]
--> /tmp/slab-0.4.5/src/lib.rs:834:20
|
834 | let ptr2 = self.entries.get_unchecked_mut(key2) as *mut Entry<T>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
And without raw pointer tagging, `cargo miri test` on `half` v1.8.2
```
error: Undefined Behavior: trying to reborrow <untagged> for Unique permission at alloc1340[0x0], but that tag only grants SharedReadOnly permission for this location
--> /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/slice/raw.rs:141:9
|
141 | &mut *ptr::slice_from_raw_parts_mut(data, len)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| trying to reborrow <untagged> for Unique permission at alloc1340[0x0], but that tag only grants SharedReadOnly permission for this location
| this error occurs as part of a reborrow at alloc1340[0x0..0x6]
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: tag was most recently created at offsets [0x0..0x6]
--> /tmp/half-1.8.2/src/slice.rs:309:22
|
309 | let length = self.len();
| ^^^^^^^^^^
help: this tag was also created here at offsets [0x0..0x6]
--> /tmp/half-1.8.2/src/slice.rs:308:23
|
308 | let pointer = self.as_ptr() as *mut u16;
| ^^^^^^^^^^^^^
```
The second suggestion is close to guesswork, but from experience it tends to be correct (as in, it tends to locate the pointer the user wanted) more often that it doesn't.
Ben Kimock [Fri, 13 May 2022 23:04:51 +0000 (19:04 -0400)]
More review feedback
* Store the local crates in an Rc<[CrateNum]>
* Move all the allocation history into Stacks
* Clean up the implementation of get_logs_relevant_to a bit
bors [Fri, 13 May 2022 19:04:59 +0000 (19:04 +0000)]
Auto merge of #2114 - cbeuw:shim-rmw, r=RalfJung
Use atomic RMW for `{mutex, rwlock, cond, srwlock}_get_or_create_id` functions
This is required for #1963
`{mutex, rwlock, cond, srwlock}_get_or_create_id()` currently checks whether an ID field is 0 using an atomic read, allocate one and get a new ID if it is, then write it in a separate atomic write. This is fine without weak memory. For instance, in `pthread_mutex_lock` which may be called by two threads concurrently, only one thread can read 0, create and then write a new ID, the later-run thread will always see the newly created ID and never 0.
```rust
fn pthread_mutex_lock(&mut self, mutex_op: &OpTy<'tcx, Tag>) -> InterpResult<'tcx, i32> {
let this = self.eval_context_mut();
let kind = mutex_get_kind(this, mutex_op)?.check_init()?;
let id = mutex_get_or_create_id(this, mutex_op)?;
let active_thread = this.get_active_thread();
```
However, with weak memory behaviour, both threads may read 0: the first thread has to see 0 because nothing else was written to it, and the second thread is not guaranteed to observe the latest value, causing a duplicate mutex to be created and both threads "successfully" acquiring the lock at the same time.
This is a pretty typical pattern requiring the use of atomic RMWs. RMW *always* reads the latest value in a location, so only one thread can create the new mutex and ID, all others scheduled later will see the new ID.
Ben Kimock [Wed, 11 May 2022 23:13:00 +0000 (19:13 -0400)]
Cleanup/Refactoring from review
* Pass a ThreadInfo down to grant/access to get the current span lazily
* Rename add_* to log_* for clarity
* Hoist borrow_mut calls out of loops by tweaking the for_each signature
* Explain the parameters of check_protector a bit more
bors [Tue, 10 May 2022 19:38:52 +0000 (19:38 +0000)]
Auto merge of #2112 - y86-dev:clarify-custom-rustc-issues, r=RalfJung
Clarified issues when building miri with a custom rustc
I came across these issues (see zulip threads [here](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/.E2.9C.94.20Changes.20not.20in.20effect) and [here](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Missing.20.2Elibrustc.2Estamp), issue [here](https://github.com/rust-lang/rust/issues/90244)) and would like to add this small bit to the docs. Feel free to change the wording.
projects / rust.git / log