]> git.lizzy.rs Git - rust.git/commit
Auto merge of #33612 - royalstream:royalstream-enc-enum-ptr, r=michaelwoerister
authorbors <bors@rust-lang.org>
Sun, 15 May 2016 12:26:50 +0000 (05:26 -0700)
committerbors <bors@rust-lang.org>
Sun, 15 May 2016 12:26:50 +0000 (05:26 -0700)
commitb3583193531c124e36aa1813cf5f8b8e667d016c
tree6dc9e451e04dffefe9d51b9b587afd96cc88631d
parent1a26d2364f9da1667ca0098d3cc4a213d131f481
parent04baf2593e782dafc979cfbfb6c9994e3a0d1e78
Auto merge of #33612 - royalstream:royalstream-enc-enum-ptr, r=michaelwoerister

gdb Pretty Print: generic encoded was failing on reference/pointer types

If you debug this program using **gdb**
```rust
fn main() {
    let x = 10;
    let y = Some(&x);
    // additional code
}
```
And you try to print **y**'s value from the debugger, you get the following:
```
(gdb) print y
Python Exception <class 'gdb.error'> Cannot convert value to int.:
$1 = {RUST$ENCODED$ENUM$0$None = Some = {0x7fff5fbff97c}}
```
What happens is that inside **debugger_pretty_printers_common.py** the method `is_null_variant` doesn't have any special handling for pointer values so it ends up calling `.as_integer()` on `discriminant_val` (which holds a pointer) and fails.

Considering it needs to handle pointers and return _true_ when the pointer is _null_, I modified the `.as_integer()` method in **gdb_rust_pretty_printing.py** to take pointers into consideration.

After this modification **gdb** prints **y** like this:
```
(gdb) print y
$1 = Some = {0x7fff5fbff97c}
```
Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.