Rollup merge of #53025 - ljedrz:debug_asserts_limited, r=varkor

Consider changing assert! to debug_assert! when it calls visit_with

The perf run from #52956 revealed that there were 3 benchmarks that benefited most from changing `assert!`s to `debug_assert!`s:

- issue #46449: avg -4.7% for -check
- deeply-nested (AKA #38528): avg -3.4% for -check
- regression #31157: avg -3.2% for -check

I analyzed their fixing PRs and decided to look for potentially heavy assertions in the files they modified. I noticed that all of the non-trivial ones contained indirect calls to `visit_with()`.

It might be a good idea to consider changing `assert!` to `debug_assert!` in those places in order to get the performance wins shown by the benchmarks.