# Kumori LSP Package

This package contains the Langium-based language implementation for the Kumori DSL. It is the part of the monorepo that knows how to parse `.kumori` files, build an AST, resolve names across files, validate models, and expose language features through Language Server Protocol handlers.

For a new developer, the useful mental model is:

1. Grammar files define the language syntax.
2. Langium generates parser and AST support from that grammar.
3. Custom services add Kumori-specific behavior such as module resolution, scoping, validation, and type checking.
4. The language server wires those services to LSP features like diagnostics, go-to-definition, references, and semantic tokens.

## Overview

This monorepo has three packages. `packages/lsp` is the language implementation package.

Its responsibilities are:

- Define the Kumori grammar in `src/language/*.langium` and `src/language/lang/*.langium`.
- Generate Langium runtime artifacts from that grammar.
- Provide Kumori-specific service implementations for workspace loading, linking, scopes, validation, and semantic tokens.
- Start the language server for Node and browser environments.

Important entry points:

- `src/main.ts`: starts the Node language server.
- `src/language/main-browser.ts`: starts the browser worker version of the language server.
- `src/language/kumori.ts`: assembles the full Langium service container.
- `langium-config.json`: tells `langium-cli` where the grammar starts and where generated files go.

## What Is Langium?

Langium is a TypeScript framework for building domain-specific languages.

At a high level, a Langium language is made of two things:

- A grammar: defines the syntax of the language.
- Services: implement behavior on top of the parsed model.

From the grammar, Langium generates core runtime pieces such as:

- AST TypeScript types.
- A parser.
- Default linking and scoping infrastructure.
- A generated dependency-injection module.
- Syntax definitions for editors.

In this package, the generator is configured in `langium-config.json`:

- Entry grammar: `src/language/kumori.langium`
- Generated output: `src/language/generated`
- TextMate syntax: `syntaxes/kumori.tmLanguage.json`
- Monaco syntax: `src/syntaxes/kumori.monarch.ts`

The main generated files developers should know about are:

- `src/language/generated/ast.ts`: AST interfaces and type guards.
- `src/language/generated/module.ts`: generated Langium DI module.
- `src/language/generated/grammar.ts`: serialized grammar at runtime.

These generated files are build artifacts. The source of truth is the grammar and custom services.

## Project Grammar

The root grammar is intentionally small:

```langium
grammar Kumori

import "lang/package"

entry Model:
    Package;
```

That means a `.kumori` file is parsed as a `Package`. The rest of the language is split into smaller grammar files under `src/language/lang`.

### What the language represents

At the grammar level, the language describes a Kumori package that can contain:

- Imports.
- Artifacts.
- A library section.

The main artifact kinds are:

- `deployment`
- `service`
- `component`
- `builtin`

This structure is defined mainly in:

- `src/language/lang/package.langium`
- `src/language/lang/artifact.langium`
- `src/language/lang/component.langium`
- `src/language/lang/service.langium`
- `src/language/lang/deployment.langium`
- `src/language/lang/builtin.langium`
- `src/language/lang/library.langium`

### Key constructs to understand first

#### Packages and imports

`package.langium` defines the top-level shape of a file:

- Optional imports.
- A package body.
- A body that can contain artifacts or a `library` block.

Imports are references to other packages and may optionally use an alias.

#### Artifacts

`artifact.langium` defines the shared structure of artifacts.

Important ideas:

- Artifact bodies are structured blocks of statements.
- Artifact statements are typed struct entries or `var` blocks.
- Non-deployment artifacts have names.
- `ArtifactName` is a reference, not just text, so artifact naming participates in linking.

#### Library definitions

`library.langium` defines reusable declarations:

- `type` definitions.
- `alias` definitions.
- `func` signatures.

This is where named types and functions enter the package-level scope.

#### Types and expressions

The language has a real expression and type grammar.

- `expression.langium` defines operator precedence, function calls, literals, lists, structs, and identifier chains.
- `struct.langium` defines typed and untyped struct entries.
- `library.langium` defines type forms such as unions, lists, structs, and type references.

The main expression forms a new developer should recognize are:

- Identifier references such as `a.b.c`.
- Function calls.
- Struct and list literals.
- Primitive literals: string, number, boolean, size.

#### Tokens and separators

`terminal.langium` defines terminals such as identifiers, strings, numeric values, and size literals. It also defines `ElementSeparator`, which allows commas or newlines in many list-like positions.

That detail matters because many Kumori constructs are intentionally newline-friendly.

## Langium Services

Langium organizes language behavior into services managed by dependency injection.

Useful built-in service categories to know:

- Parser: turns source text into an AST.
- Scope computation: precomputes symbols visible in a document.
- Scope provider: answers "what names are visible here?".
- Linker: resolves cross-references.
- Index manager: tracks exported symbols across documents.
- Workspace manager: loads and rebuilds documents in a workspace.
- Document builder: runs the build pipeline and validation phases.
- Document validator: produces diagnostics.
- Semantic token provider: powers semantic syntax highlighting.

In this project, service wiring happens in `src/language/kumori.ts`.

`createKumoriServices` merges three layers for shared services and three layers for language-specific services:

1. Langium defaults.
2. Langium-generated Kumori services.
3. Custom Kumori services.

There are two service groups:

- Shared services: one instance for workspace-wide concerns.
- Language services: one instance for Kumori-specific language behavior.

### How they are wired here

The custom shared module contributes:

- `workspace.WorkspaceManager -> KumoriWorkspaceManager`
- `workspace.IndexManager -> KumoriIndexManager`
- `workspace.DocumentBuilder -> KumoriDocumentBuilder`
- `references.KumoriModules -> KumoriModules`
- `lsp.DocumentUpdateHandler -> KumoriDocumentUpdateHandler`

The custom language module contributes:

- `references.ScopeComputation -> KumoriScopeComputation`
- `references.ScopeProvider -> KumoriScopeProviver`
- `references.Linker -> KumoriLinker`
- `references.KumoriPackages -> KumoriPackages`
- `validation.KumoriValidations -> KumoriValidations`
- `validation.KumoriTypeSystem -> KumoriTypeSystem`
- `validation.DocumentValidator -> KumoriDocumentValidator`
- `lsp.SemanticTokenProvider -> KumoriSemanticTokenProvider`

After service creation, `createKumoriServices` also:

- Registers the Kumori service set in Langium's service registry.
- Registers custom validation checks through `registerValidationChecks`.

## Custom Services In This Project

This section lists the custom services actually implemented in this package and where they fit.

### Shared services

#### `KumoriWorkspaceManager`

File: `src/language/kumori-workspace.ts`

Responsibilities:

- Initializes workspace folders.
- Loads builtin standard-library documents from `src/language/builtin`.
- Discovers Kumori modules in the workspace.
- Resolves dependent modules before building documents.
- Adds all module documents to Langium and triggers the initial build.

This is the main reason the package understands more than a single file.

#### `KumoriIndexManager`

File: `src/language/kumori-index-manager.ts`

Responsibilities:

- Extends Langium's default index invalidation logic.
- Marks files in the same directory as affected when one file changes.

In practice, this makes package-local updates reindex more aggressively, which fits the way Kumori packages are organized.

#### `KumoriDocumentBuilder`

File: `src/language/kumori-builder.ts`

Responsibilities:

- Extends the default document build pipeline.
- Preserves diagnostics that originate from validation work on other documents.
- Tracks every document that ended up with diagnostics.

This matters because validation in Kumori can cross document boundaries.

#### `KumoriModules`

File: `src/language/kumori-module.ts`

Responsibilities:

- Stores known modules and caches module lookups.
- Resolves package references against a module manifest and dependencies.
- Tracks dependency resolution results.
- Invalidates documents belonging to a module when manifests change.

This is the core module-resolution service used by package and import logic.

#### `KumoriDocumentUpdateHandler`

File: `src/language/UpdateHandler.ts`

Responsibilities:

- Extends file watching so the server also watches `kumori.mod.json`.
- Reloads module metadata when manifests change.
- Re-registers module contents before forwarding normal document updates.
- Filters manifest files out before parsing, because they are not Kumori source files.

This service connects filesystem changes to the module model.

### Language services

#### `KumoriScopeComputation`

File: `src/language/kumori-scope.ts`

Responsibilities:

- Computes exported symbols for artifacts and library declarations.
- Computes local scopes inside artifacts and struct-like bodies.
- Adds special local names such as `self`, `var`, and `interface`.
- Assigns synthetic names to deployments, which are anonymous in source.

This is the precomputation step that builds symbol tables per document.

#### `KumoriScopeProviver`

File: `src/language/kumori-scope.ts`

Responsibilities:

- Decides which names are visible for a specific reference.
- Builds scope layers from local scope, imports, current package, header package, and stdlib.
- Handles import-specific behavior and prefixed imported names.
- Resolves chained identifier access by switching scope to the referenced target.

If you are debugging name resolution, this is one of the first files to read.

#### `KumoriLinker`

File: `src/language/kumori-link.ts`

Responsibilities:

- Resolves references using Kumori's custom scope rules.
- Supports lazy loading of referenced AST nodes.
- Chooses between implementation and header artifacts when multiple matches exist.
- Special-cases `ArtifactName` so those references prefer header files.
- Supports both single references and multi-references.

This is where scope results become actual linked AST references.

#### `KumoriPackages`

File: `src/language/kumori-package.ts`

Responsibilities:

- Maps a node or document to its containing package directory.
- Resolves imported packages through `KumoriModules`.
- Falls back to builtin packages when an import resolves to stdlib content.
- Collects all documents that belong to a package.

This service gives the rest of the language layer a package-level view of the workspace.

#### `KumoriValidations`

File: `src/language/kumori-validations.ts`

Responsibilities:

- Creates the validation context passed into individual validator implementations.
- Works with `registerValidationChecks` to register all validators with Langium.

This is not a validator by itself. It is the service-level entry point for the validation subsystem.

#### `KumoriTypeSystem`

File: `src/language/kumori-types.ts`

Responsibilities:

- Exposes type inference, assignability checking, evaluation, and completion.
- Manages caches for those operations through a reusable context.
- Wraps the lower-level implementation in `src/language/type-system`.

This is the core semantic engine behind many validations and semantic tokens.

#### `KumoriDocumentValidator`

File: `src/language/kumori-builder.ts`

Responsibilities:

- Extends Langium's default validator.
- Attaches the originating document to diagnostics so cross-file errors can be reported on the correct file.

It exists specifically to support the custom cross-document diagnostic flow.

#### `KumoriSemanticTokenProvider`

File: `src/language/kumori-semantic.ts`

Responsibilities:

- Provides semantic token types for identifiers and literals.
- Uses the type system to distinguish types, functions, structs, and variables.
- Highlights import aliases, parameters, struct keys, literals, and special identifiers.

This is the main custom editor-facing feature implementation in the language service module.

### Related non-service customization

Two important pieces are not DI services but are still part of the package's language behavior:

- `src/language/lsp/start.ts`: custom language-server startup that replaces Langium's default diagnostics handler so diagnostics from related documents are also published.
- `src/language/validations/*`: actual validation rules, grouped by domain such as artifact, component, deployment, service, and builtin rules.

The validation registry is generated in `src/language/validations/_registry.ts`, and `registerValidationChecks` registers those validators by AST node type.

## Validation Layout

The validation system is intentionally modular.

- Each rule lives in its own file under `src/language/validations`.
- `_registry_gen.sh` generates `_registry.ts` during the build.
- `kumori-validations.ts` converts that registry into Langium validation checks.

For a new contributor, this means:

- Add or modify a validation rule in `src/language/validations/...`.
- Let the build regenerate the registry.
- Do not hand-edit `_registry.ts`.

## How Everything Fits Together

The runtime flow is:

1. A `.kumori` file is parsed using the grammar from `src/language/kumori.langium` and the imported grammar fragments.
2. Langium builds AST nodes defined in `src/language/generated/ast.ts`.
3. `KumoriWorkspaceManager` loads workspace documents, builtin documents, and module dependencies.
4. `KumoriScopeComputation` precomputes local and exported symbols.
5. `KumoriScopeProviver` and `KumoriLinker` resolve names and references across the current package, imported packages, header packages, and stdlib.
6. `KumoriTypeSystem` provides semantic operations used by validators and semantic highlighting.
7. `KumoriDocumentValidator` and the generated validation registry produce diagnostics.
8. `src/language/lsp/start.ts` registers LSP handlers so editors can request completions, definitions, references, highlights, folding ranges, semantic tokens, and diagnostics.

If you want a practical order for reading the code, start here:

1. `src/language/kumori.ts`
2. `src/language/kumori.langium`
3. `src/language/lang/package.langium`
4. `src/language/kumori-workspace.ts`
5. `src/language/kumori-scope.ts`
6. `src/language/kumori-link.ts`
7. `src/language/kumori-types.ts`
8. `src/language/kumori-validations.ts`

That path gives a good top-down view: syntax first, then document loading, then name resolution, then semantic analysis.