bors [Sun, 20 Mar 2022 02:37:09 +0000 (02:37 +0000)]
Auto merge of #1975 - DrMeepster:backtrace_fix, r=RalfJung
Make backtraces work with #[global_allocator]
Currently, backtraces break when the global allocator is overridden because the allocator will attempt to deallocate memory allocated directly by Miri.
~~This PR fixes that by using a new memory kind and providing a function to deallocate it. We can't call the custom allocator to allocate because it's not possible to call a function in the middle of a shim.~~
This PR fixes that by adding a new version of the backtrace API accessible by setting `flags` to 1. Existing code still functions.
bors [Fri, 18 Mar 2022 20:38:37 +0000 (20:38 +0000)]
Auto merge of #2024 - saethlin:better-local-check, r=RalfJung
Consider the cargo workspace when checking if a frame is local
`DefId::is_local` returns a result which is technically correct, but doesn't match the user's intuition when running integration tests or doctests. This incorporates the workspace crates mentioned in `cargo metadata` into the check for whether a frame is local to match user intuition.
For example, here is the backtrace you get from `MIRIFLAGS=-Zmiri-tag-raw-pointers cargo miri test` in `bytes` 1.1.0:
```
--> /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/slice/raw.rs:131:14
|
131 | unsafe { &mut *ptr::slice_from_raw_parts_mut(data, len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ trying to reborrow for Unique at alloc67158, but parent tag <untagged> does not have an appropriate item in the borrow stack
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
= note: inside `std::slice::from_raw_parts_mut::<u8>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/slice/raw.rs:131:14
= note: inside `bytes::bytes::rebuild_boxed_slice` at /tmp/bytes-1.1.0/src/bytes.rs:938:19
= note: inside closure at /tmp/bytes-1.1.0/src/bytes.rs:904:18
= note: inside `<std::sync::atomic::AtomicPtr<()> as bytes::loom::sync::atomic::AtomicMut<()>>::with_mut::<[closure@bytes::bytes::promotable_even_drop::{closure#0}], ()>` at /tmp/bytes-1.1.0/src/loom.rs:17:17
= note: inside `bytes::bytes::promotable_even_drop` at /tmp/bytes-1.1.0/src/bytes.rs:895:5
= note: inside `<bytes::Bytes as std::ops::Drop>::drop` at /tmp/bytes-1.1.0/src/bytes.rs:515:18
= note: inside `std::ptr::drop_in_place::<bytes::Bytes> - shim(Some(bytes::Bytes))` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
note: inside `copy_to_bytes_less` at tests/test_buf.rs:112:1
--> tests/test_buf.rs:112:1
|
112 | }
| ^
note: inside closure at tests/test_buf.rs:106:1
--> tests/test_buf.rs:106:1
|
105 | #[test]
| ------- in this procedural macro expansion
106 | / fn copy_to_bytes_less() {
107 | | let mut buf = &b"hello world"[..];
108 | |
109 | | let bytes = buf.copy_to_bytes(5);
110 | | assert_eq!(bytes, &b"hello"[..]);
111 | | assert_eq!(buf, &b" world"[..])
112 | | }
| |_^
= note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)
```
We get these because the integration tests are occurring in a crate called `test`, not the actual `bytes` crate. With this PR, we get this:
```
= note: inside `std::slice::from_raw_parts_mut::<u8>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/slice/raw.rs:131:14
note: inside `bytes::bytes::rebuild_boxed_slice` at /tmp/bytes-1.1.0/src/bytes.rs:938:19
--> /tmp/bytes-1.1.0/src/bytes.rs:938:19
|
938 | Box::from_raw(slice::from_raw_parts_mut(buf, cap))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure at /tmp/bytes-1.1.0/src/bytes.rs:904:18
--> /tmp/bytes-1.1.0/src/bytes.rs:904:18
|
904 | drop(rebuild_boxed_slice(buf, ptr, len));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `<std::sync::atomic::AtomicPtr<()> as bytes::loom::sync::atomic::AtomicMut<()>>::with_mut::<[closure@bytes::bytes::promotable_even_drop::{closure#0}], ()>` at /tmp/bytes-1.1.0/src/loom.rs:17:17
--> /tmp/bytes-1.1.0/src/loom.rs:17:17
|
17 | f(self.get_mut())
| ^^^^^^^^^^^^^^^^^
note: inside `bytes::bytes::promotable_even_drop` at /tmp/bytes-1.1.0/src/bytes.rs:895:5
--> /tmp/bytes-1.1.0/src/bytes.rs:895:5
|
895 | / data.with_mut(|shared| {
896 | | let shared = *shared;
897 | | let kind = shared as usize & KIND_MASK;
898 | |
... |
905 | | }
906 | | });
| |______^
note: inside `<bytes::Bytes as std::ops::Drop>::drop` at /tmp/bytes-1.1.0/src/bytes.rs:515:18
--> /tmp/bytes-1.1.0/src/bytes.rs:515:18
|
515 | unsafe { (self.vtable.drop)(&mut self.data, self.ptr, self.len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= note: inside `std::ptr::drop_in_place::<bytes::Bytes> - shim(Some(bytes::Bytes))` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
note: inside `copy_to_bytes_less` at tests/test_buf.rs:112:1
--> tests/test_buf.rs:112:1
|
112 | }
| ^
note: inside closure at tests/test_buf.rs:106:1
--> tests/test_buf.rs:106:1
|
105 | #[test]
| ------- in this procedural macro expansion
106 | / fn copy_to_bytes_less() {
107 | | let mut buf = &b"hello world"[..];
108 | |
109 | | let bytes = buf.copy_to_bytes(5);
110 | | assert_eq!(bytes, &b"hello"[..]);
111 | | assert_eq!(buf, &b" world"[..])
112 | | }
| |_^
= note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)
```
Note that this kind of inflation is rather rare to see. Most backtraces change not at all or only a tiny bit.
I originally implemented this to support another improvement to Miri diagnostics, but I think this is hairy enough to deserve its own PR, if somewhat poorly-motivated.
bors [Thu, 17 Mar 2022 13:26:29 +0000 (13:26 +0000)]
Auto merge of #1971 - saethlin:sb-details, r=RalfJung
Add a lot more information to SB fatal errors
In fatal errors, this clarifies the difference between a tag not being present in the borrow stack at all, and the tag being present but granting SRO. It also introduces a little notation for memory ranges so we can mention to the user that the span may point to code that operates on multiple memory locations, but we are reporting an error at a particular offset.
This also gets rid of the unqualified phrase "the borrow stack" in errors, and clarifies that it is the borrow stack _for some location_.
The crate `pdqselect` v0.1.1:
Before:
```
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no item granting read access to tag <2357> at alloc1029 found in borrow stack.
```
After:
```
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| attempting a read access using <2357> at alloc1029[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc1029[0x0..0x4]
```
And the crate `half` v1.8.2
Before:
```
131 | unsafe { &mut *ptr::slice_from_raw_parts_mut(data, len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ trying to reborrow for Unique at alloc1051, but parent tag <2091> does not have an appropriate item in the borrow stack
```
After:
```
131 | unsafe { &mut *ptr::slice_from_raw_parts_mut(data, len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| trying to reborrow <2091> for Unique permission at alloc1051[0x0], but that tag only grants SharedReadOnly permission for this location
| this error occurs as part of a reborrow at alloc1051[0x0..0x6]
```
Ben Kimock [Sun, 6 Feb 2022 00:44:56 +0000 (19:44 -0500)]
Print more in SB error diagnostics
This tries to clarify exactly why an access is not valid by printing
what memory range the access was over, which in combination with
tag-tracking may help a user figure out the source of the problem.
bors [Tue, 8 Mar 2022 12:54:18 +0000 (12:54 +0000)]
Auto merge of #2006 - RalfJung:readme-ci, r=RalfJung
update recommended CI snippet, add GHA example
If this snippet works as expected in https://github.com/rust-lang/portable-simd/pull/255 (which we can test tomorrow), then this adjusts our README to that nicer script.
bors [Sat, 5 Mar 2022 22:47:51 +0000 (22:47 +0000)]
Auto merge of #1970 - asquared31415:open_unix_varargs, r=RalfJung
Allow varargs for libc::open when it is allowed by the second argument
This PR allows `libc::open` to be called using two or three arguments as defined in https://man7.org/linux/man-pages/man2/open.2.html
The presence of the third argument depends on the value of the second argument. If the second argument dictates that the third argument is *required* miri will emit an error if the argument is missing. If the second argument does *not* require a third argument, then the argument is ignored and passed as 0 internally (it would be ignored by libc anyway)
bors [Sat, 26 Feb 2022 22:17:04 +0000 (22:17 +0000)]
Auto merge of #1989 - saethlin:short-backtraces, r=RalfJung
Only print the pruning note if the trace was definitely pruned
Per https://github.com/rust-lang/miri/pull/1987#discussion_r815337994
The only problem with this is that it looks like we can't emit the note about `-Zmiri-backtrace=full` if we also emit a tag-tracking diagnostic. The final note about the setting simply doesn't appear on the final error if it is printed before. I feel like we're running up against some diagnostic deduplication logic?
bors [Sat, 26 Feb 2022 08:56:02 +0000 (08:56 +0000)]
Auto merge of #1987 - saethlin:short-backtraces, r=oli-obk
Prune stacktraces for tag-tracking diagnostics too
In https://github.com/rust-lang/miri/pull/1977 we forgot to use the pruning logic when printing tag-tracking messages. I just saw this output before this PR which is very silly:
```
test sanity ... note: tracking was triggered
--> src/lib.rs:110:21
|
110 | unsafe { &*(&self.value[range] as *const [T] as *const Aligned<A, [T]>) }
| ^^^^^^^^^^^^^^^^^^ created tag 160954
|
= note: inside `<Aligned<A4, [u8]> as std::ops::Index<std::ops::RangeTo<usize>>>::index` at src/lib.rs:110:21
note: inside `sanity` at src/lib.rs:261:30
--> src/lib.rs:261:30
|
261 | let y: &Aligned<_, _> = &y[..2];
| ^^^^^^
note: inside closure at src/lib.rs:229:1
--> src/lib.rs:229:1
|
228 | #[test]
| ------- in this procedural macro expansion
229 | / fn sanity() {
230 | | use core::mem;
231 | |
232 | | let x: Aligned<A2, _> = Aligned([0u8; 3]);
... |
284 | | let _: &[u8] = y;
285 | | }
| |_^
= note: inside `<[closure@src/lib.rs:229:1: 285:2] as std::ops::FnOnce<()>>::call_once - shim` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `test::__rust_begin_short_backtrace::<fn()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:575:5
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:566:30
= note: inside `<[closure@test::run_test::{closure#1}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1854:9
= note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
= note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside `test::run_test_in_process` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:598:18
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:492:39
= note: inside `test::run_test::run_test_inner` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:530:13
= note: inside `test::run_test` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:562:28
= note: inside `test::run_tests::<[closure@test::run_tests_console::{closure#2}]>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:305:17
= note: inside `test::run_tests_console` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/console.rs:286:5
= note: inside `test::test_main` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:116:15
= note: inside `test::test_main_static` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:135:5
= note: inside `main`
= note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:122:18
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:145:18
= note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
= note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:128:48
= note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside `std::rt::lang_start_internal` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:128:20
= note: inside `std::rt::lang_start::<()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:144:17
= note: this note originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)
error: Undefined Behavior: trying to reborrow for SharedReadOnly at alloc61593+0x2, but parent tag <160954> does not have an appropriate item in the borrow stack
--> src/lib.rs:261:30
|
261 | let y: &Aligned<_, _> = &y[..2];
| ^^^^^^ trying to reborrow for SharedReadOnly at alloc61593+0x2, but parent tag <160954> does not have an appropriate item in the borrow stack
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
= note: inside `sanity` at src/lib.rs:261:30
note: inside closure at src/lib.rs:229:1
--> src/lib.rs:229:1
|
228 | #[test]
| ------- in this procedural macro expansion
229 | / fn sanity() {
230 | | use core::mem;
231 | |
232 | | let x: Aligned<A2, _> = Aligned([0u8; 3]);
... |
284 | | let _: &[u8] = y;
285 | | }
| |_^
= note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)
note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace