From: bors[bot] <26634292+bors[bot]@users.noreply.github.com> Date: Sun, 12 Dec 2021 16:38:56 +0000 (+0000) Subject: Merge #10995 X-Git-Url: https://git.lizzy.rs/?a=commitdiff_plain;h=db2a7087b994e20f264f26ad6db75184282ad120;p=rust.git Merge #10995 10995: internal: switch from trait-based TokenSource to simple struct of arrays r=matklad a=matklad cc #10765 The idea here is to try to simplify the interface as best as we can. The original trait-based approach is a bit over-engineered and hard to debug. Here, we replace callback with just data. The next PR in series will replace the output `TreeSink` trait with data as well. The biggest drawback here is that we now require to materialize all parser's input up-front. This is a bad fit for macro by example: when you parse `$e:expr`, you might consume only part of the input. However, today's trait-based solution doesn't really help -- we were already materializing the whole thing! So, let's keep it simple! Co-authored-by: Aleksey Kladov --- db2a7087b994e20f264f26ad6db75184282ad120