More information for mbe

[rust.git] / docs / dev / architecture.md

diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md
index 2ba8b7967d00242bbaaabde93240470c3e43c23c..c5e4acea31e3659b7b3bb4b5f66d573932bf6a3a 100644 (file)
--- a/docs/dev/architecture.md
+++ b/docs/dev/architecture.md
@@ -75,7 +75,7 @@ Original [libsyntax parser](https://github.com/rust-lang/rust/blob/6b99adeb11313
 
 **Architecture Invariant:** the parser is independent of the particular tree structure and particular representation of the tokens.
 It transforms one flat stream of events into another flat stream of events.
-Token independence allows us to pares out both text-based source code and `tt`-based macro input.
+Token independence allows us to parse out both text-based source code and `tt`-based macro input.
 Tree independence allows us to more easily vary the syntax tree implementation.
 It should also unlock efficient light-parsing approaches.
 For example, you can extract the set of names defined in a file (for typo correction) without building a syntax tree.
@@ -254,6 +254,22 @@ A single `rust-analyzer` process can serve many projects, so it is important tha
 These crates implement macros as token tree -> token tree transforms.
 They are independent from the rest of the code.
 
+`tt` crate defined `TokenTree`, a single token or a delimited sequence of token trees.
+`mbe` crate contains tools for transforming between syntax trees and token tree.
+And it also handles the actual parsing and expansion of declarative macro (a-la "Macros By Example" or mbe).
+
+For proc macros, the client-server model are used to run proc-macro.
+We pass an argument `--proc-macro` to `rust-analyzer` binary to start a separate process  (`proc_macro_srv`).
+And the client (`proc_macro_api`) provides an interface to talk to that server separately.
+
+And then token trees are passed from client, and the server will load the corresponding dynamic library (which built by `cargo`).
+And due to the fact the api for getting result from proc macro are always unstable in `rustc`,
+we maintain our own copy (and paste) of that part of code to allow us to build the whole thing in stable rust.
+ 
+ **Architecture Invariant:**
+Bad proc macros may panic or segfault accidentally. So we run it in another process and recover it from fatal error.
+And they may be non-deterministic which conflict how `salsa` works, so special attention is required.
+
 ### `crates/cfg`
 
 This crate is responsible for parsing, evaluation and general definition of `cfg` attributes.
@@ -373,8 +389,48 @@ There's no additional checks in CI, formatting and tidy tests are run with `carg
 
 **Architecture Invariant:** tests do not depend on any kind of external resources, they are perfectly reproducible.
 
+### Error Handling
+
+**Architecture Invariant:** core parts of rust-analyzer (`ide`/`hir`) don't interact with the outside world and thus can't fail.
+Only parts touching LSP are allowed to do IO.
+
+Internals of rust-analyzer need to deal with broken code, but this is not an error condition.
+rust-analyzer is robust: various analysis compute `(T, Vec<Error>)` rather than `Result<T, Error>`.
+
+rust-analyzer is a complex long-running process.
+It will always have bugs and panics.
+But a panic in an isolated feature should not bring down the whole process.
+Each LSP-request is protected by a `catch_unwind`.
+We use `always` and `never` macros instead of `assert` to gracefully recover from impossible conditions.
+
 ### Observability
 
-I've run out of steam here :)
 rust-analyzer is a long-running process, so its important to understand what's going on inside.
-We have hierarchical profiler (`RA_PROFILER=1`) and object counting (`RA_COUNT=1`).
+We have several instruments for that.
+
+The event loop that runs rust-analyzer is very explicit.
+Rather than spawning futures or scheduling callbacks (open), the event loop accepts an `enum` of possible events (closed).
+It's easy to see all the things that trigger rust-analyzer processing, together with their performance
+
+rust-analyzer includes a simple hierarchical profiler (`hprof`).
+It is enabled with `RA_PROFILE='*>50` env var (log all (`*`) actions which take more than `50` ms) and produces output like:
+
+```
+85ms - handle_completion
+    68ms - import_on_the_fly
+        67ms - import_assets::search_for_relative_paths
+             0ms - crate_def_map:wait (804 calls)
+             0ms - find_path (16 calls)
+             2ms - find_similar_imports (1 calls)
+             0ms - generic_params_query (334 calls)
+            59ms - trait_solve_query (186 calls)
+         0ms - Semantics::analyze_impl (1 calls)
+         1ms - render_resolution (8 calls)
+     0ms - Semantics::analyze_impl (5 calls)
+```
+
+This is cheap enough to enable in production.
+
+
+Similarly, we save live object counting (`RA_COUNT=1`).
+It is not cheap enough to enable in prod, and this is a bug which should be fixed.