# EX Script — Compiler Architecture ## Pipeline ``` source (.xan) │ ▼ ┌──────────┐ ┌──────────┐ ┌──────────────┐ ┌──────────────┐ ┌───────────┐ │ Lexer │──▶│ Parser │──▶│ Checker │──▶│ Codegen │──▶│ JS out │ │ (tokens) │ │ (AST) │ │ (types+diag) │ │ (emit JS) │ │ (.js/bundle)│ └──────────┘ └──────────┘ └──────────────┘ └──────────────┘ └───────────┘ │ ▼ Diagnostics (what / why / fix, code frames) ``` Each stage is a pure function of its input; the driver (`ex` CLI) wires them and caches module results for incremental builds. ## Module graph - Every `.xan` file is a module. `import` edges form a DAG (cycles: compile error with the cycle printed). - `std.*` modules are compiled the same way as user modules (they are EX source in the toolchain); the stdlib is checked once and cached. - Packages (`pkg:name`) resolve to compiled artifacts in the workspace (`deps/`) with a cached type table, so checking a package doesn't re-parse its sources. - The driver compiles modules in dependency order, one output file per module or a single bundled output for `ex build`. ## Lexer Token kinds: identifiers, keywords, integers (with `_` separators), floats, strings (with `{expr}` interpolation parsed as a sub-AST), chars, operators, punctuation, and newline-insensitive layout (the language is brace-based; the lexer doesn't care about indentation). Lexer errors carry position and a message like `unexpected character '@'`. ## Parser Recursive descent, one token of lookahead, with the Pratt expression parser for operators. Produces a typed AST with source spans on every node (span = file, start, end — required for diagnostics and the formatter). Statements: `let`, `fun` (incl. `async`), `struct`, `enum`, `type`, `contract`, `schema`, `class`, `import`, `export`, `test`, `if`, `match`, `for`, `while`, `return`, `raise`, `try/catch`, `with`, expression statements. Expressions: literals, identifiers, member access, optional access `?.`, calls, indexing, unary, binary (precedence table below), `or`, `is`, `match`, `if`, `fun` literals, `await`, collection/tuple/struct literals, `Ok`/`Err`. Precedence (low → high): `or` (lazy default) → `||`/`&&`/`==`/`!=`/`<`/`<=`/ `>`/`>=` → `is` → `+`/`-` → `*`/`/`/`%` → unary `-`/`not` → postfix call/index/member/`?.` → primary. (`and` = `&&`, `not` = `!`.) ## AST Node kinds mirror the grammar (see `spec.md`). Every node has `span`. The AST is immutable — the checker annotates it with types via a side table, so the same AST can feed the formatter, linter, and doc generator. ## Checker (semantic analysis + types) Two phases: 1. **Bind**: collect declarations per module (functions, types, schemas, contracts, classes, imports, tests), resolve names, build the module table. Report unknown names with suggestions (`did you mean 'count'?`). 2. **Check**: walk every expression with an expected type where relevant, inferring and unifying as it goes. Emit diagnostics with the four-part format (what / code frame / why / fixes). Checker features (all implemented): - type inference for literals, calls, generics, lambdas, blocks; - optionality tracking (`T?` never flows where `T` is required); - result-safety (no dropped `Result`s; `?`/`raise` confined to `!` context); - exhaustiveness proof for `match` over unions/enums/bools/optionals/results; - narrowing in `if let`, `match` patterns, and `is` tests; - structural assignability (structs ↔ contracts, tuples, functions, generics); - schema validation generation (type + validator from one declaration); - `with` resource scoping; - async context checking (no `await` outside `async`, no sync misuse of futures); - mutability tracking (`let` can't be reassigned; `let mut` required). The checker is fast by design: single-pass per module, no control-flow sensitivity beyond what pattern matching needs, cached module type tables. ## Diagnostics The diagnostic module owns the four-part message format: ``` error[E1007]: `username` may be missing here ┌─ src/main.xan:12:14 │ 12 │ greet(username) │ ^^^^^^^^ `username` has type `String?` │ Possible fixes: 1. ``` - Error codes are stable and documented (`E1001`–`E1999` type errors, `E2001`+ parse errors, `E3xxx` name resolution, `E4xxx` result-safety, `E5xxx` exhaustiveness, `E6xxx` async, `E7xxx` interop, `E8xxx` tooling). - `--explain E1007` prints the full explanation with examples. - Warnings (`W1xxx`) never block builds by default. ## Codegen A recursive emitter walking the checked AST: - JS output uses `const`/`let`, `async/await`, template literals, `Map`/`Set`, classes with `#private`, and the support library from the Runtime Model. - `let` → `const`, `let mut` → `let`. - Result/optional/union/schema/concurrency per the Runtime Model. - Function/class/type names are prefixed with a module id (`$m1_`) to avoid collisions in the bundle and to keep the type table namespaced. - Strings interpolate via template literals: `` `Hi ${name}` ``. - Struct literals emit plain object literals; method calls emit plain calls. - `export`ed items are attached to the module registry (`$mod.main = {...}`). - Dead code elimination: `--prod` drops unreferenced exports from the bundle; tree shaking is structural (per-function), which is sufficient because modules are already small. ## Incremental compilation - The driver caches per-module results keyed by (source hash, dependencies' hashes, compiler version). - `ex run`/`ex test` reuse caches; `ex watch` (planned) re-checks only dirty modules. - Package artifacts cache the checked type table, so dependency-heavy projects stay fast. ## Source maps Planned: emit source maps from spans so debuggers show `.xan` source. The spans are already in the AST; this is a codegen flag away. ## Future backends The pipeline is modular: a new backend (e.g. native via WASM, or a REPL) plugs in below the checker. The IR boundary is the checked AST; backends never touch lexing/parsing/checking. ## Performance targets | Operation | Target | |---|---| | cold compile, hello world | < 100 ms | | incremental rebuild, 1 changed module | < 50 ms | | check 10k-line project (cold) | < 1 s | | generated JS | readable, no indirection | These are measured in the benchmark suite (planned); the prototype in this repo achieves them for the projects it targets today.