]> git.lizzy.rs Git - rust.git/commit - src/tools/miri
Rollup merge of #81856 - Smittyvb:utf16-warn, r=matthewjasper
authorDylan DPC <dylan.dpc@gmail.com>
Sat, 27 Feb 2021 20:56:15 +0000 (21:56 +0100)
committerGitHub <noreply@github.com>
Sat, 27 Feb 2021 20:56:15 +0000 (21:56 +0100)
commitbe3d1eb3010b48f5b0512fc83cc029bb321fb3ab
tree6260c72ef61516da8803cb6ca2d1cc56aa34856d
parent94736c434ee154b30e2ec22ec112b79e3f6c5884
parented8c68644c9a352f61c3b4591b6fc18653e2ffc2
Rollup merge of #81856 - Smittyvb:utf16-warn, r=matthewjasper

Suggest character encoding is incorrect when encountering random null bytes

This adds a note whenever null bytes are seen at the start of a token unexpectedly, since those tend to come from UTF-16 encoded files without a [BOM](https://en.wikipedia.org/wiki/Byte_order_mark) (if a UTF-16 BOM appears it won't be valid UTF-8, but if there is no BOM it be both valid UTF-16 and valid but garbled UTF-8). This approach was suggested in https://github.com/rust-lang/rust/issues/73979#issuecomment-653976451.

Closes #73979.