--- type: standard status: active scope: core appliesTo: - core/tools/** - operator/extensions/tools/** --- # Tooling Standard Canonical contract for shared and library tools on the platform. This document is authoritative for every tool that ships in `core/tools/`, and the default contract for tools under `operator/extensions/tools/`. Operators MAY extend this standard with additional, more-restrictive rules in their own `operator/knowledge/engineering/standards/tooling.md`; they MUST NOT relax it. A tool that does not satisfy every section below is non-compliant. Compliance is enforced by code review, by the tooling lint described in [§17](#17-lint-enforcement), and by the readers of every README who treat it as the spec. See also: [`naming.md`](./naming.md) (when present in your tree), the per-tool README under `core/tools//README.md`, and the broader principles document at `operator/knowledge/principles.md` if your operator layer ships one. --- ## 1. Identity & naming - Reverse-domain identity in `package.json` under `"name"`: `..`, e.g. `ai.example.platform.svc`. - `` is the functional domain (`platform`, `ops`, `knowledge`, …). - `` is a short identifier matching the CLI alias. - Short CLI alias; single lowercase word, verb-ish or noun-ish. The alias is the directory name under `core/tools/` (or `operator/extensions/tools/`) and the name of the shim in `bin/`. - Location: `core/tools//`. The bin shim is `core/tools//bin/`. - Reference implementations shipped in core: `core/tools/secrets/`, `core/tools/memory-sync/`, `core/tools/smoke/`, `core/tools/distill/`. ## 2. Language and cross-platform posture - **Tools are written in Node.js. Full stop.** Shell scripts (`.sh`, `.ps1`, `.bat`, `.zsh`, `.fish`) are not acceptable tool implementations. The platform ships to macOS, Linux, and Windows; a Unix-shell tool silently breaks on Windows; a PowerShell tool silently breaks on Unix; a platform that depends on maintaining two parallel shell implementations is a platform with twice the surface area and half the confidence. - **The only acceptable shell files in the repo are thin launchers at repo root** (`dave.sh` + `dave.ps1`) which exist because *something* has to pre-date Node being available. Any logic inside those launchers that grows beyond "parse flags, find node, hand off" belongs in a Node.js tool they invoke. - Tools shell out to external **binaries** (git, docker, ssh, sops, age, node itself) as needed; those are cross-platform programs with native installers, not scripts. `node:child_process.spawn` does this fine without a shell in the middle. - Rationale: every `.sh`-only tool ever shipped on this platform created a Windows gap that wasn't noticed until an operator hit it. The cost of writing one tool in Node is always less than the cost of discovering the gap in production. ## 3. Dependencies - **Zero runtime npm dependencies in the tool itself.** Tools use only the Node standard library. - Allowed stdlib surface: `node:util` (especially `parseArgs`), `node:child_process`, `node:fs/promises`, `node:path`, `node:os`, `node:test`, `node:assert/strict`. - No schema libraries, no CLI frameworks, no test frameworks, no lodash. - Rationale: - **Auditability**; the entire tool is readable without third-party spelunking. - **Supply-chain hygiene**; no transitive footguns, no lockfile churn. - **Reproducibility**; a cloned repo runs immediately; no `npm install` step to fail. - **Longevity**; the Node stdlib outlives every fashion in userland. - Runtime infrastructure that a tool *manages* (e.g. pm2 for native service supervision, docker for container orchestration) is a separate concern; those are external programs the tool shells out to, not dependencies of the tool. ## 4. Project structure Canonical layout: ``` core/tools// bin/ # node shebang shim; one line package.json # name, version, type: module, no dependencies README.md # the spec (see §9) src/ cli.mjs # arg parsing, verb routing, exit-code translation {domain}.mjs # domain loaders (e.g. discover.mjs, manifest.mjs) runner.mjs # SOLE chokepoint for external effects (see §7) commands/ {verb}.mjs # thin glue per command renderers/ {runtime}.mjs # pure functions per target runtime test/ fixtures/ # sample inputs, fake trees *.test.mjs # node:test, one per module under test ``` Per-directory rationale: - **`bin/`**; a single-line node shebang shim importing `src/cli.mjs`. The entry point is unambiguously the CLI module; the shim carries no logic. - **`src/cli.mjs`**; the only module that knows about `process.exit` codes. Commands return numbers or throw; `cli.mjs` translates to exit codes. - **`src/{domain}.mjs`**; pure loaders, validators, and domain helpers. No spawning, no SSH. - **`src/runner.mjs`**; the chokepoint (§7). Nothing else spawns processes. - **`src/commands/`**; glue. Each command file is thin: discover → render → run → format. - **`src/renderers/`**; pure functions per runtime. No I/O. Easy to test, easy to swap. - **`test/`**; `node:test` files alongside `fixtures/`. No separate test config. Smaller tools may compress this layout; a single `src/cli.mjs` is fine when there is no chokepoint to isolate. Larger tools must observe the boundaries above. ## 5. CLI conventions - **Verb-noun pattern**: ` list`, ` status `, ` health `. Verb first, target after. - **`--help` / `-h`** prints usage and exits `0`. - Arg parsing uses `node:util.parseArgs` with `strict: true`. No ad-hoc string splitting. - Subcommand routing is an explicit `switch` in `cli.mjs`. No dynamic dispatch, no plugin mechanism. - Unknown verbs exit `1` with usage printed to stderr. - Flags use long form primarily (`--device foo`). Short flags only for `--help` / `-h`. ## 6. Exit code contract Every tool uses the same four exit codes: | Code | Meaning | |---|---| | `0` | OK | | `1` | Usage error (bad arguments, unknown target, ambiguous selector) | | `2` | Logical / operational failure (the thing you asked about is broken, unhealthy, missing) | | `3` | Transport / IO failure (ssh, spawn, network, disk) | Rationale: automation can distinguish *"you invoked me wrong"* from *"the world is broken"* from *"I couldn't even reach the world"*. Translation is centralised in `cli.mjs`; domain code throws errors carrying an exit-code property (e.g. `err.exitCode`), and the CLI wrapper maps them to `process.exit`. ## 7. Module boundaries and the chokepoint pattern **Every external-effect API has exactly one place where it can happen.** This is a general principle, not a tool-specific detail. When an operation has safety properties that must always hold; rate limits, destructive guards, dry-run support, retry logic, auditing; those properties are enforced in a single chokepoint module. Scattered call sites mean scattered violations. - A canonical example: an SSH-driven tool exposes a `runner.mjs` that is the only module that spawns processes or opens SSH. It deliberately does **not** expose a "run a command N times remotely" API. Tools that need more remote work must build a bigger script and dispatch it once. - Renderer-per-runtime separation: `renderers/{runtime}.mjs` produces the script or config for a given substrate. Renderers are pure; no I/O. Commands drive the actual execution via the runner. - Domain concerns (`discover.mjs`, `manifest.mjs`) are separate from execution concerns (`runner.mjs`) are separate from presentation concerns (`renderers/`, `cli.mjs`). When designing a new tool, ask: *what operation, if it ever happens in the wrong place or the wrong way, causes the most damage?* That operation gets a chokepoint. ## 8. Testing - Test runner: `node:test`. Assertions: `node:assert/strict`. Nothing else. - Layout: `test/*.test.mjs` plus `test/fixtures/` for sample inputs. - Run: `node --test core/tools//test/*.test.mjs`. - **Dependency injection over mocking libraries.** Modules that touch the outside world expose a seam for tests; e.g. `runner.mjs` accepts (or can be monkey-patched to use) a fake spawner. No `sinon`, no `jest.mock`. - Coverage expectations: - Every module has a test file. - Commands are tested via fake renderers. - Renderers are pure; tested directly as input-to-output functions. - The runner is exercised against a real local command *and* against a fake spawner to verify framing and error paths. - Real network / SSH never appears in the test suite. ## 9. README as contract The README is the specification. It is not marketing, not a tutorial, not a changelog. Read-and-extend: a new contributor should be able to read only the README and understand the tool completely. Required sections: - **Identity**; reverse-domain name, one-line purpose. - **Status**; what's implemented vs deferred. - **Install / invoke**; how to run it. - **Commands**; every verb, synopsis, flags. - **Exit codes**; the contract from §6. - **Operational discipline rules**; non-negotiable constraints baked into the architecture (see §10). - **Project layout**; the directory tree with one-line rationale per entry. - **Tests**; how to run them. - **The tooling standard**; back-reference to this document, naming the sections that govern the tool. Tone: dense, declarative, no fluff. This is a contract. ## 10. Operational discipline rules Some tools have non-negotiable safety rules baked into their architecture. These rules are documented in the README, enforced in the chokepoint module, and have no escape hatch API. **Example; SSH discipline.** A target host enforces, say, a 6-connection / 60-second SSH rate limit. Exceeding it locks the agent out and trips a firewall dead-man switch. Therefore every command of an SSH-driven tool performs at most one SSH call per cluster. For a `health` verb, all healthchecks are concatenated into a single bash script and dispatched once via `ssh bash -s`. The runner deliberately has no "run remote command N times" API; there is no remote-loop footgun because there is no remote-loop API. When a tool has a similar constraint (rate limits, destructive operations, audit requirements), it follows the same pattern: one chokepoint, no loop API, rule written into the README. ## 11. JSON conventions All JSON produced or consumed by tools uses **camelCase** property names. **No exceptions.** This matches the rule the Librarian enforces across the knowledge system. Values that are identifiers (service IDs, role IDs, map keys) keep their natural form; only the property *names* are governed. Snake_case and kebab-case property names are violations and must be corrected in place; consumers are updated in the same change. ## 12. Reference implementations Concrete tools shipped in core that exemplify this standard: - [`core/tools/secrets/`](../../tools/secrets/); vault CLI; clean chokepoint, hand-rolled validation, exit-code discipline. - [`core/tools/memory-sync/`](../../tools/memory-sync/); bidirectional cache mirror; demonstrates the `2`-overload for "asked-thing-not-where-it-should-be" semantics. - [`core/tools/smoke/`](../../tools/smoke/); health-check orchestrator; pure renderer + chokepoint runner pattern. - [`core/tools/distill/`](../../tools/distill/); scheduled tool with the audit-record contract from §15. Read these before designing a new tool. ## 13. Versioning Tools are versioned via the `version` field in their `package.json`. Release notes live in the README under a `## Changelog` heading, or in a dedicated `CHANGELOG.md` once the history grows large enough to warrant it. SemVer applies: the CLI surface and exit-code contract are the public API. ## 14. Installing on PATH Tools live under `core/tools//` and are invoked through their `bin/` shim. To make them callable as a bare command from any directory, the convention is: - A single directory `core/tools/bin/` at the repo root contains **symlinks** to each tool's `bin/` shim. - The PATH addition is **one entry**, set by the operator (e.g. in `~/.zshrc` or via the launcher script). Adding a new tool means dropping one symlink; no PATH change. - The shim is `#!/usr/bin/env node` with one line: `import('../src/cli.mjs').then(m => m.main(process.argv.slice(2)))`. Node resolves the symlink to its target before resolving the relative `import`, so the symlink Just Works without any `--preserve-symlinks` ceremony. To install on a new machine: ```bash ln -sf ..//bin/ core/tools/bin/ ``` That's the entire install procedure. Operators who automate this can add the PATH entry to whatever shell-rc loader they ship. ## 15. Scheduled-tool audit-record contract Any scheduled tool that writes a `runs.jsonl` MUST append one record on every non-dry-run exit path; success, legitimate no-op, AND failure. Downstream freshness monitors (e.g. `tasks status` overlays) read `runs.jsonl`'s last line; if any exit path skips the append, the monitor sees stale data and either alarms falsely or stays silent on real outages. **Required fields:** | Field | Values | Notes | |---|---|---| | `status` | `ok`, `noop-*`, `fail-*` | Closed vocabulary. Prefix match for monitors: `noop-*` = fresh, `fail-*` = failing | | `ts` | RFC-3339 (`YYYY-MM-DDTHH:MM:SSZ`) | Must `new Date(ts)` without NaN. Filename-safe ts with dashes belongs only in filenames and commit-message slugs | **Rules:** 1. Append on every non-dry-run exit path; success, no-op, AND failure. 2. Skip the append on dry-run (no side effects). 3. Tests must cover every exit path's audit record, not just the happy path. 4. Backwards compat: consumers that fall back to `exitCode` when `status` is absent handle historical records. ## 16. Operational artefacts must be gitignored Per-host operational files; audit logs (`runs.jsonl`), last-run timestamps, host-local caches, PID files; **must be in `.gitignore`**. They must never be git-tracked. **Why:** Any tool that combines (a) writes to a tracked file on every invocation (audit log, counter, timestamp) with (b) a precondition that fails on working-tree modifications will deadlock itself within one cycle. The first run dirties the tree; the next run sees the dirty tree and refuses to start; the refusal writes another audit line; the tree gets dirtier. This is not a hypothetical. Multiple scheduled tools across the platform have hit exactly this deadlock; `runs.jsonl` was tracked, the runner had a clean-tree precondition, and the schedule wedged for days. **Rule:** cross-host visibility of operational state is a legitimate need, but git-tracking the file is the wrong solution. Use a workspace endpoint, a sync job, or shared storage instead. The file stays local and gitignored; the visibility layer reads it on demand. ## 17. Lint enforcement A `core/*` document, role definition, or tool source file must not refer to paths under `operator/`, `agents/`, `work/`, `projects/`, `.vault/`, or `schedules/`. Those directories are operator-owned; nothing in core is allowed to depend on them. Cross-layer references invert the layering and silently break for forks that have not populated their operator layer in the same shape as the originating fork. The `core/tools/lint-core-isolation/` tool enforces this rule. It scans `core/**/*.{md,mjs,js,ts,json}` and exits `2` with a list of violations on any cross-layer reference. CI runs the lint on every push and pull request. Allowed exceptions are limited and explicitly enumerated in the lint tool's README: - Comments or strings naming `operator/` purely as a *destination convention* (e.g. server code that locates the operator's knowledge tree at runtime). These are functional references, not documentation pointers, and they live in the lint's allowlist with a one-line justification. - Test fixtures that simulate the operator layer for the purpose of testing core code. These live under `core/tools//test/fixtures/operator/` and the lint allowlist names that prefix. If you need a new exception, change the lint allowlist and explain in the same change why the reference cannot be eliminated. "It would be inconvenient to remove" is not a justification. --- ## Operator extensions Operators may layer additional rules on top of this standard at `operator/knowledge/engineering/standards/tooling.md`. The operator-side document SHOULD: - Cite this document as the base. - State only the operator-specific *additions* (e.g. operator-private CLI conventions, operator-private chokepoints, naming for operator-private runtime targets). It MUST NOT contradict any rule in this document. Tools under `operator/extensions/tools/` follow this standard as the floor; the operator's extensions raise it where needed.