Rollup merge of #51658 - oli-obk:unregress_perf, r=nikomatsakis

Only do sanity check with debug assertions on

r? @nnethercote

I'm slighty confused. These changes address code that the `unused-warnings` benchmark doesn't go through, yet I see a 5% improvement to nightly on the `check` run, and no improvement on the other runs.

Maybe this change allows unrelated code in the same function to be better optimized?