# Expert Council [English](README.md) | [简体中文](README.zh-CN.md) Expert Council is a local, multi-model, cost-aware expert orchestration system for Pi and MCP hosts such as Codex. It discovers the models connected through Pi's registered LLM APIs and coding plans, combines runtime metadata with user-defined billing policy, capability profiles, and local reliability data, dynamically assembles a small semantic expert team, executes bounded tasks through Pi, and returns compact structured results to the Main Agent. Key advantages: | Advantage | Description | |---|---| | Cost savings | Makes flexible use of subscribed plans and LLM APIs, automatically matching the most suitable model to each task's difficulty | | Speed | Runs multiple suitable models concurrently, accelerating repository exploration and context compression | | Safer | Assigns different read-only/writable permissions per expert; writable experts commit into an isolated Git worktree that the Main Agent reviews before integration | | Context savings | The Main Agent no longer carries long tool-call-heavy context; it receives summarized, structured results from experts | In practice, the theoretically strongest model is not automatically the best executor. A model with stable tool calls, reliable shell behavior, and low marginal cost can deliver more real task value than a stronger but less dependable one. We recommend configuring a cost-effective, execution-strong model as the Main Agent; when a complex problem appears, Expert Council can dispatch strong-reasoning models for review or planning. ## Current status The current version (0.8.6) includes: - **Interactive experts**: a running expert can pause on a major, hard-to-reverse, or ambiguous decision and present 2-4 recommended options (+ optional free text) to the Main Agent through `request_decision`; the host answers with `expert_respond` and the expert continues in the **same session**. Non-terminal — distinct from `report_and_stop`. - **Dynamic tool permissions**: preset role tools are a **seed, not a ceiling**. An expert requests a missing tool via `request_tool`; the host grants `once` (auto-revoked after one use), `persistent` (this session), or `reject`. `security.toolGrants` gives operator-defined persistent per-role grants. A read-only execution can never be escalated to a mutating/shell tool (isolation guarantee). - **Interaction discovery on a correctness channel**: an open interaction surfaces as `pendingInteraction` in `expert_status(view:"running")` and `expert_result(includeProgress)`, and the native Pi package additionally wakes the host with an `expert-council-interaction` notice the moment one opens; headless hosts (Codex/MCP, no server-push) keep polling. Bounded per execution (default 3 rounds + wait timeout). - **Cross-language environments**: `provisionWorkspace` is no longer Node-only. A driver registry materializes node/pnpm/yarn/bun, python (uv/poetry/host venv), rust, go, jvm/maven, dotnet, ruby, php, elixir from each toolchain's already-global download cache. Environment-as-code backends (`flake.nix`, `.devcontainer/`) are detected and delegated, not re-implemented. New `security.workspaceProvisioning.strategy` (`auto`/`drivers`/`as-code`/`in-place`) and `runtimeEnv` (`isolated`/`host-env`); concurrent worktrees never share a recompiled target directory. - **Progress observability toggle**: `security.observability.expertWindow` selects how much of an expert's live work is visible - `off` (default, nothing ambient), `events` (a bounded live progress block in the running views), or `interactive`, which additionally writes a **live event stream the operator can follow from a second terminal** with `expert-council watch --exec ID --follow`, at zero Main Agent context cost. `security.observability.streamToHost` can switch the streamed progress off, and `redactToolArgs` (default `true`) decides whether tool calls are recorded by name only. None of these gate the `pendingInteraction` correctness channel, nor an explicit `expert_result(includeProgress)` snapshot. - **Struggle detection**: `security.guardrails` counts tool failures and budget consumption the runtime actually observed, surfaces them as `attention` in the running views plus one native host notice per warning, steers the expert at most twice per execution, and never aborts a run. `maxTotalWallMs` caps the aggregate wall clock across all attempts of one delegation, and `executionMetadata.attemptHistory` explains what each attempt did. - A host-agnostic Core: config validation, model normalization, per-model/per-provider billing multipliers, profile layering, role scoring, task classification, dynamic team sizing, retry/escalation, and telemetry aggregation. - An execution runtime built on Pi's current `ModelRuntime` and `createAgentSession` APIs. - Per-expert hard tool allowlists and installed-Skill filtering. - Detached Git worktree isolation for writable experts, plus **automatic dependency provisioning** from the repository lockfile (npm/pnpm/bun/yarn, plus uv/poetry/cargo and other drivers), with script suppression where the toolchain supports it, an allowlisted child environment, per-execution worktree reuse, and direct host `.venv` interpreter paths surfaced for Python ecosystems. - A **runtime verification gate**: after provisioning, the repository typecheck and tests run automatically; a failing gate downgrades a successful result to `partial` and engages the corrected-retry path — zero Main Agent overhead. - **`report_and_stop` expert tool**: when an expert determines the task cannot be completed (missing tool, absent environment, denied permission), it submits a structured stop report — blocker, findings, risks, `recommendedNextAction` — delivered as a `partial` result that terminates the delegation loop without retry or escalation, with the mutation worktree retained for inspection. - **Host-bound expert lifetime** (`security.expertLifetime`, default `host-bound`): running experts are aborted automatically when the host session quits or is replaced, so orphaned experts never burn quota with no receiver; `detached` restores the old behavior. - **Failure results keep artifacts**: on writable runs, timed-out, session-error, and thrown-error results include `filesChanged` and the last assistant text, so a long timeout no longer returns an empty result. Read-only runs correctly report no `filesChanged` at all. A `partial` result from a **read-only** role is now delivered rather than re-run (defect #32): the council will not spend a second full attempt on a retry the Main Agent is better placed to judge, so it returns the incomplete answer with a risk line saying so. A worker whose run is partial still escalates. An expert that called `report_and_stop` is unchanged by this rule - it ends the delegation without a second attempt, as it always has, because it declared the task blocked rather than merely incomplete. - **State write-side clamp**: oversized arrays and text are truncated at the single persistence choke point, so a verbose expert can no longer write an unreadable state file. - **`expert_availability_reset`**: clear misrecorded or recovered availability markers by `*`/provider/`provider/id` without editing files or restarting. - **`expert_verify`**: run a bounded command plugin-side in a retained worktree or validated workspace and get the real exit code and output tail; `tests[]` now carry exit codes, counts, durations, and output tails. - **`expert_status` views**: a bounded `summary` default (running with remaining budget, recent completions, provider slots), `running`, and legacy `full` — cutting observation-context cost. - **Reliable completion notifications**: the package retries the finish message briefly on the idle/streaming race instead of dropping it. - `timeoutMs` and `reasoningLevel` as required delegation arguments, chosen deliberately per task and model — on a batch every entry carries its own pair, so only single delegations read them from the top level; compositions can pin per-role/per-model reasoning levels that override the argument. - Expert fail-fast discipline: when the assigned tools or workspace make a task impossible, experts stop immediately with a structured `missing_context`/`permission_error` result that is relayed in real time instead of burning retries. - Persistent council compositions: named rosters in `council-compositions.json` (roles may pin multiple models and reasoning levels), a first-build composition menu, session bindings, `model` pinning, and concurrent same-role dispatch. - Provider limits and concurrency: per-provider daily/weekly weighted-token caps (accounted in `usage-ledger.json` via `costMultiplier`) and concurrency limits from `route-policy.json`; excluded candidates report the reason. - Tiered runtime availability markers: dead models (24h), plan-quota exhaustion (6h, provider-wide), and **transient TPM/RPM throttling (2 minutes)** — throttling is no longer misread as quota exhaustion. - `council-config.json` operator configuration discovered by default in the data directory (no environment variable); `expert_inspect` returns its path, the effective provisioning mode, and the effective progress-visibility settings so the Main Agent can edit them on request. - A CLI with JSON output. - An MCP Server with 13 async semantic tools, event-driven completion waits, a feedback loop, and explicit worktree cleanup. - A native Pi Package. - Provider session-error pass-through: upstream refusals such as `403 AccessDenied` reach the Main Agent with real diagnostics and the correct failure type. - Cross-process shared model assessment: availability markers are visible to parallel instances without restarts. - A self-contained Codex plugin bundling the shared Skill, a stdio MCP Server, and the verified Pi SDK runtime, discovering workspaces through the Codex host's own `codex/sandbox-state-meta` capability (no hooks or global SDK resolution). - Deterministic automated tests that never spend model quota. ## Architecture ```text Codex or Pi Main Agent | | semantic tools / shared Skill v Expert Council Core - resource and model normalization - billing and capability profiles - deterministic routing - roles and team sizing - retry and escalation - telemetry aggregation | v Pi Runtime - callable model discovery - hard tool allowlists - installed-Skill filtering - bounded expert sessions - workspace isolation / | \ CLI Pi Package MCP Server | Codex plugin ``` TypeScript project references enforce that dependencies flow only in these directions: ```text core <- pi-runtime <- cli <- mcp-server <- codex-integration <- pi-package ``` Core does not import Pi, Codex, MCP transport, filesystem, shell, or process APIs. The CLI, MCP Server, and host distributions all use the same service and routing logic. ## Quick start Requirements: - Node.js 22.19 or later. - npm 11 or a compatible version. - Pi installed and configured with at least one usable model. - When writable experts need worktree isolation, the Git repository must have at least one commit. ### Install the Pi Package Install from npm (recommended): ```bash pi install npm:@expert-council/pi-package pi list pi --verbose ``` `pi list` should show `npm:@expert-council/pi-package` and its resolved directory; a newly started verbose Pi session should load `dist/extension.js`, the `expert-council` Skill, and the twelve semantic tools. To upgrade later: ```bash pi update npm:@expert-council/pi-package ``` ### Install the Codex plugin (optional) To run Codex as the Main Agent, install the pinned prebuilt plugin directly from its Git marketplace; no repository clone or local build is required: ```bash codex plugin marketplace add Labiey/expert-council-router --ref v0.8.6 --json codex plugin add expert-council@expert-council-router --json ``` Fully restart Codex Desktop after installation. The plugin ships its own tested Pi SDK runtime, so the globally installed `pi` package is not used at runtime; Pi still must have been installed and configured once — or provider credentials placed manually under `~/.pi` — so the account and model catalog are available. See [Codex plugin](#codex-plugin) for Windows CLI discovery, verification, upgrade, and removal instructions. ### Build from source (development) ```bash npm install npm run build npm test ``` Discover models without calling any of them: ```bash node packages/cli/dist/bin.js models --json node packages/cli/dist/bin.js inspect --json ``` Assemble an expert team without executing any expert: ```bash node packages/cli/dist/bin.js build "fix the device hot-swap race condition" --max-experts 4 --json ``` Delegate only when you are sure actual Pi models should be called: ```bash node packages/cli/dist/bin.js delegate architecture-oracle "analyze the concurrent invocation path" \ --workspace /path/to/repo --timeout-ms 600000 --reasoning-level high --json ``` Zero configuration uses conservative capability defaults, marks unverifiable billing types as `unknown`, and refuses non-isolated writes. It never guesses that an API is free and never infers model quality from a model name. ## Model discovery `PiExpertRuntime` calls Pi's `ModelRuntime.getAvailable()` instead of using a hardcoded catalog. A model that merely exists in a registry or user profile is not routed to; only models Pi reports as currently callable are used. The runtime normalizes: - provider and model ID; - display name; - reasoning support and the reasoning-level map exposed by Pi; - context and maximum output windows; - input modalities; - published API price fields; - safe compatibility metadata. The runtime resolves a locally installed compatible Pi SDK first, then an explicit `PI_CODING_AGENT_MODULE` directory, then a compatible global npm Pi installation. If all fail, it returns an actionable diagnostic instead of fabricating a model list. Pi builds this inventory once per session, and provider catalogs can keep stale model names, so `listAvailableModels()` may include a model the upstream can no longer serve. Routing treats such a model as callable until a real attempt fails; this is why failures with dead-model evidence mark the model unavailable in the persisted model assessment (see the routing section below). Markers expire after 24 hours, and a recovered model is retried automatically. ## Configuration Specify a configuration file through the `EXPERT_COUNCIL_CONFIG` environment variable or the CLI's `--config PATH` flag. Start from [`config/examples/balanced.example.json`](https://github.com/Labiey/expert-council-router/blob/main/config/examples/balanced.example.json). Profile precedence: ```text built-in conservative defaults < user configuration or optional presets < current-task runtime overrides ``` Objective runtime metadata is merged separately. Local outcome data influences routing only after at least three samples accumulate, and the adjustment is bounded by `routing.localLearningMaxAdjustment`. Explicit user configuration always wins. ### Billing policy Supported billing types: ```text subscription metered quota free unknown ``` Marginal cost is expressed as a single numeric `costMultiplier` because published token prices cannot express subscription plans, fixed quotas, local inference, or promotional credits. The Pi Runtime adapter classifies runtime-reported subscription access or named Token Plan catalogs as `subscription`; otherwise a provider whose Pi model catalog exposes non-zero prices is `metered`, and providers without reliable evidence stay `unknown`. `expert_inspect` returns the inference source, and explicit user configuration always has the highest priority. Models within the same metered provider are still compared on specific prices through `routing.apiPriceWeight` (default `0.35`); an all-zero price table is treated as "not provided", never guessed to be free. **Per-model billing entries.** Subscription token plans carry periodic quotas (often weekly) with per-model burn rates, so one provider-level weight cannot express the real marginal cost. Add `provider/id` keys to override the provider default for specific models — routing checks the explicit `billingProfile` first, then the model-level entry, then the provider-level entry. The same works in `model-assessment.json` billing and in user configuration: ```json { "billing": { "providers": { "subscription-provider": { "billingType": "subscription", "costMultiplier": 0.1 }, "subscription-provider/qwen3.8-max": { "billingType": "subscription", "costMultiplier": 2.0 }, "scarce-provider": { "billingType": "quota", "costMultiplier": 5.0 } } } } ``` ### Billing multipliers `costMultiplier` is a relative token-consumption weight used for cost scoring and provider cap accounting. It defaults to `1.0` when omitted and must be within `0.01`–`100`. Lower is more cost-efficient for cost-weighted roles; the cost-efficiency score is `10 / (1 + costMultiplier)` before the billing-type bonus. Reference values: metered flash-class models ≈`1.0`, metered flagship full-size ≈`5.0`, token-plan flash-class ≈`0.1`, token-plan flagship ≈`2.0`. The removed tier vocabulary is still accepted and mapped deterministically so existing configuration keeps working: | Legacy `marginalCostClass` | `costMultiplier` | |---|---| | `very-low` | `0.1` | | `low` | `0.5` | | `normal` | `1.0` | | `high` | `3.0` | | `scarce` | `5.0` | The legacy `usagePreference` field is ignored and dropped; it no longer affects routing. ### Operator config (`council-config.json`) `council-config.json` is the operator configuration file. It is optional and discovered by default in the shared data directory (the same folder as `model-assessment.json` and `route-policy.json`) — no environment variable is required. Resolution order: 1. `configPath` option or `EXPERT_COUNCIL_CONFIG` environment variable — highest precedence, and the file **must** exist; 2. `/council-config.json` — read when present, silently skipped when absent; 3. otherwise everything runs on built-in defaults (`workspaceProvisioning.mode = "none"`). The file accepts the full operator schema: `security`, `billing`, `profiles`, and `routing`. Typical example: The complete standard form, with every key at its built-in default. **Every key is optional** - a file may contain one line, and anything absent falls back to the value below: ```json { "billing": { "providers": {} }, "profiles": { "models": {} }, "routing": { "maxExperts": 4, "minimumWorkerToolReliability": 4, "localLearningMaxAdjustment": 1, "apiPriceWeight": 0.35, "roleWeights": {}, "diversity": { "repeatedModelPenalty": 0.35, "reviewerSameProviderPenalty": 0.25, "reviewerSameFamilyPenalty": 0.5 }, "taskClassification": { "tinyMaxWords": 8, "tinyMaxCjkChars": 18, "complexMinWords": 35, "complexMinCjkChars": 60, "complexSignalThreshold": 2 } }, "retry": { "maxAttempts": 3, "maxEscalations": 2, "correctedRetriesPerModel": 1 }, "security": { "workspaceStrategy": "auto", "allowInPlaceMutations": false, "allowedWorkspaceRoots": [], "trustedSkills": [], "worktreeRetentionMs": 86400000, "toolGrants": {}, "expertLifetime": "host-bound", "observability": { "expertWindow": "off", "streamToHost": true, "redactToolArgs": true, "autoOpenWindow": false, "recordReasoning": false, "contentStream": "none", "contentByRole": {}, "contentWindowLines": 10, "contentWindowChars": 120, "contentEventBytes": 65536, "contentFileBytes": 10485760, "contentTotalBytes": 209715200 }, "guardrails": { "warnHost": true, "nudgeExpert": true, "consecutiveToolFailures": 3, "minCallsForRatio": 8, "failureRatio": 0.5, "budgetFractions": [0.6, 0.85] }, "workspaceProvisioning": { "mode": "none", "strategy": "auto", "runtimeEnv": "isolated", "timeoutMs": 600000, "maxConcurrent": 1, "scrubEnv": true, "removalTimeoutMs": 300000 } } } ``` Provider and model keys are `"provider"` and `"provider/id"` exactly as `expert_inspect` reports them. A filled-in variant of the two records that most people actually edit: ```json { "billing": { "providers": { "acme-plan": { "billingType": "subscription", "costMultiplier": 0.1 }, "acme-metered": { "billingType": "metered", "costMultiplier": 1 }, "acme-host": { "billingType": "quota", "costMultiplier": 5, "disabled": false } } }, "profiles": { "models": { "acme-plan/acme-flash": { "reasoning": 7, "planning": 7, "architecture": 6.5, "coding": 7, "debugging": 7, "review": 6.5, "longContext": 7.5, "toolReliability": 7, "bashReliability": 7, "autonomousExecution": 7, "speed": 8.5, "preferredReasoningByRole": { "scout": "low", "architecture-oracle": "high" }, "incompatibleRoles": [], "billingProfile": "acme-plan", "disabled": false }, "acme-metered/acme-pro": { "coding": 9, "debugging": 9, "toolReliability": 9, "bashReliability": 9, "autonomousExecution": 9 } } } } ``` Capability scores are 0-10 and every dimension is optional (omit what you do not want to assert); `null` means "no opinion, use the conservative default". The ten billing types are `subscription`, `metered`, `quota`, `free`, `unknown`; `costMultiplier` is a relative token-weight, not a currency amount. `incompatibleRoles` is a hard veto, and `overrideUnavailableMarker` lets an operator overrule a marker the runtime learned from a failure. Field notes: - `security.workspaceProvisioning.mode` — `"none"` (default, no installs), `"auto"` (lockfile-detected install per ecosystem), or `"custom"` (runs `command` verbatim). - `security.workspaceProvisioning.verifyCommand` — a single command as a flat argv array, run after a provisioned mutation expert finishes instead of the default typecheck-then-test pair. - `security.workspaceProvisioning.scrubEnv` — when `true` (default), provisioning and verification children receive an allowlisted environment only; `~/.npmrc` registry tokens can still reach the child (documented residual). - `security.worktreeRetentionMs` — how long provisioned worktrees are retained for review before the retention prune (default 24h). - `billing` / `profiles` / `routing` — the same schemas as `model-assessment.json` billing entries, model capability profiles, and role weights. **Main-Agent editing contract**: `expert_inspect` returns the file location and the effective provisioning mode under `operatorConfig`. When the user asks to change provisioning behavior, the Main Agent edits the file directly and tells the user a host session restart is required for the change to apply. A malformed JSON or invalid enum is a hard error at startup (an operator typo never silently disables a security setting). ### Worktree provisioning Mutation worktrees start from committed `HEAD` and therefore contain no untracked local artifacts. The runtime can provision them from the repository's own committed lockfile before the expert runs; it is opt-in and inert by default: ```json { "security": { "workspaceProvisioning": { "mode": "auto", "timeoutMs": 600000, "maxConcurrent": 1, "scrubEnv": true, "removalTimeoutMs": 300000 } } } ``` This configuration lives in `council-config.json` inside the shared data directory (the same folder as `model-assessment.json`). The file is optional and discovered by default — no environment variable is required: if it exists it is read, if not everything stays at the defaults. `expert_inspect` returns its location under `operatorConfig` so the Main Agent can edit it on the user's behalf (changes apply after the host session restarts). An explicit `configPath` option or `EXPERT_COUNCIL_CONFIG` environment variable still takes precedence and must point at an existing file. - `mode` is `none` (default, never provision), `auto` (detect the committed lockfile and install), or `custom` (run `command` verbatim as an argv array). - `auto` runs `pnpm install --frozen-lockfile --prefer-offline`, `npm ci --prefer-offline --no-audit --no-fund`, or `bun install --frozen-lockfile`, each with `--ignore-scripts`. Yarn is the deliberate exception: Yarn Berry removed that CLI flag, so passing it would break the install rather than harden it, and the council instead sets `YARN_ENABLE_SCRIPTS=false` and `YARN_IGNORE_SCRIPTS=true` in the child environment, which covers both Yarn generations. A committed driver registry also materializes python (`uv sync`, `poetry install`, or a host `.venv` interpreter), rust, go, jvm/maven, dotnet, ruby, php, and elixir from each toolchain's own global download cache; those drivers do not all offer an equivalent switch, so each one declares whether it can suppress build scripts, and provisioning through a driver that cannot reports a limitation to the Main Agent instead of claiming otherwise; a repository that declares an environment-as-code backend (`flake.nix`, `.devcontainer/`) is detected and delegated to, never re-implemented. - Only mutation worktrees are provisioned; read-only roles run in the main workspace and are never provisioned. - The child environment is allowlist-scrubbed (`PATH`, `HOME`, `USERPROFILE`, `APPDATA`, `LOCALAPPDATA`, `TEMP`, `TMP`, `SYSTEMROOT`, `SYSTEMDRIVE`, `COMSPEC`, `PROGRAMFILES`, `PROGRAMDATA`, `GIT_*`, `npm_config_registry`, `npm_config_cache`); API tokens and cloud credentials are not forwarded. A driver's own suppression settings are merged **after** scrubbing, so they survive the allowlist and a parent `YARN_ENABLE_SCRIPTS=true` cannot override them. - When provisioning is `ready`, the runtime runs `verifyCommand` if configured, otherwise `npm run typecheck` then `npm test`. A failing gate downgrades a successful expert result to `partial` with `failureType: "test_failure"`. ### Observability and the expert window `security.observability` decides how much of an expert's live work is visible. It is deliberately separate from the `pendingInteraction` channel, which is a correctness surface and is therefore always reported: ```json { "security": { "observability": { "expertWindow": "interactive", "streamToHost": true, "redactToolArgs": true } } } ``` | `expertWindow` | What you get | | --- | --- | | `off` (default) | Nothing ambient: running views show elapsed and remaining budget only. | | `events` | Plus a bounded live progress block (`messageCount` and the last assistant text) in `expert_status` running views, when `streamToHost` is true. | | `interactive` | Plus a live event stream the operator can follow from **another terminal**, which costs the Main Agent no context at all. | The stream is one JSON object per line in `/observability/.jsonl`: `started`, `tool_started`, `tool_finished`, `assistant_text`, `interaction_opened`, `interaction_answered`, then exactly one of `stopped` / `completed` / `failed`. Follow it with: ```bash expert-council watch --exec exec_abc123 --follow ``` - `--follow` always terminates: on a terminal event, when `--timeout-ms` (default 300000) elapses, or if the file disappears. It tails by byte offset and never prints a partially written line. `--json` emits raw objects instead of formatted lines. - Expert narration is collapsed onto one bounded line per event; there is no tool output and no chain of thought on disk. - `redactToolArgs` (default `true`) records tool invocations by name only; set `false` to also write a bounded argument summary, which contains whatever the expert passed - file paths and full command lines. Only do this where a local file holding those values is acceptable. - Stream files are pruned after 7 days. - The stream is written by the process running the experts. An expert dispatched from your Pi session is followed by the `watch` command reading the same data directory, not by an unrelated checkout. - `expert_inspect` reports `runtimeCapabilities.eventStream`, so a Main Agent can tell whether the requested tier is actually available; `interactive` on a runtime that cannot write a stream degrades to `events` and says so in `warnings`. - `watch` closes when the council writes a delegation-level terminator (`delegation_final`), and reports the close as `delegation finished` rather than the first attempt's terminal event. That distinction exists because a retried or escalated delegation emits **one terminal event per attempt**: a watcher that stopped at the first would close on the failure and never show the attempt that actually answered the task. A stream from an older runtime, or one whose process died, instead closes only after it has been silent for a generous period (15s by default, tunable with `--quiet-ms`), because silence is not death: an expert thinks between tool calls for seconds and a build can go quiet for minutes. That fallback applies **once a terminal event has been seen**; a stream that never reached one is bounded by `--timeout-ms` instead, since a fresh expert can legitimately say nothing while the model thinks. `--timeout-ms` always bounds the wait. - Every event carries the attempt number that produced it, and a retried attempt renders as `[role model #2]`, so an escalation is visible in the window itself. Event rendering is clamped to one line, because a second line from a single event would desynchronise the operator's tail. ### Opening an observer window automatically Nothing is opened on your behalf by default. `security.observability.autoOpenWindow` opts in to a terminal of its own per delegation, running exactly the command above - the same tail you would start by hand, so the Main Agent still spends no context on it: ```json { "security": { "observability": { "expertWindow": "interactive", "autoOpenWindow": true } } } ``` - **One window per delegation, not per attempt.** A retried or escalated attempt reuses the same execution id and the same stream file, and would otherwise flash a second window showing half of the work. The window is released when the delegation reaches its final marker. - **No limit on simultaneous windows.** Three experts in flight is three windows; the count is the operator's business, not the council's. - **The expert finishing does not close the window.** The follower stops at `delegation_final` and then waits for a keypress, so you can scroll back over the whole session and dismiss it yourself. - **The window's own deadline follows the expert's budget** (1.5x, floored at 15 minutes, capped at the CLI's 60-minute ceiling), so the observer can never expire before the work it is watching. - **The window is view-only.** Decision prompts and tool approvals still surface in the host, so there is exactly one place that can answer an expert. - **Only this configuration key opens a window.** No tool argument, expert output, or task text can trigger one, and task content never reaches the command line: the arguments are the execution id, the role, and numeric options. Opening a window is the one place where the runtime starts a process on the operator's behalf: it launches the operating system's own terminal host (Windows Terminal when installed, otherwise the classic console) running this project's own CLI follower. When the terminal host or the CLI cannot be found, or the platform is not Windows, no window appears, no delegation is affected, and `expert_inspect` / `expert_status` report a single limitation line explaining the fallback. To point the observer at a specific CLI build, set `EXPERT_COUNCIL_CLI`; to force a terminal host, set `EXPERT_COUNCIL_TERMINAL`. The manual route stays available for everyone: run `expert-council watch --exec --follow` from any second terminal, which is also how you re-read a finished delegation, since the stream file remains on disk until it is pruned after 7 days. ### What the observer records (content dials) By default the stream records **names and counters**: which tool ran, whether it worked, how many calls it took. It never records what the tool returned. That is a deliberate exposure choice, and `security.observability.contentStream` is how you widen it - in five ascending dials, so you pick the risk rather than inheriting it from a checkbox: | Dial | What reaches disk | Why you would choose it | | --- | --- | --- | | `none` (default) | names, counters, outcome markers | nothing extra leaves the process | | `assistant` | assistant text, as it is produced | follow the reasoning-as prose without shipping file contents | | `assistant+tool-tail` | the above plus the **tail** of each tool result | see the failing line without storing whole files | | `transcript` | the above plus the **whole** tool result (behind a visible seam) | reproduce what the expert saw, after the fact | | `transcript+args` | the above plus full tool **arguments** | shell commands and paths too - the highest value for debugging, and the only dial that can record a secret | ```json { "security": { "observability": { "expertWindow": "interactive", "autoOpenWindow": true, "contentStream": "transcript", "contentByRole": { "debugger": "transcript", "reviewer": "assistant" }, "contentWindowLines": 10, "contentWindowChars": 120, "contentEventBytes": 65536, "contentFileBytes": 10485760, "contentTotalBytes": 209715200 } } } ``` - **The dial is resolved as**: built-in `none` < `contentStream` < `contentByRole[role]` < `EXPERT_COUNCIL_CONTENT`. Configuration and environment only - no tool argument, expert output, or task text can turn recording on, because this is a privacy switch, not a performance one. A role name in `contentByRole` is validated against the real role list, so a typo is an error naming the legal values rather than a dial that quietly never applies. - **Off unless you ask for it**: `thinking` / `reasoning` parts. Set `security.observability.recordReasoning: true` (or `EXPERT_COUNCIL_REASONING=1` for one process) *and* a dial that records assistant text. Reasoning is then stored on the same cursors and ceilings, always marked `reasoning: true` and shown as `thinks` rather than `says`, and the runtime names the switch in its capability limitations. The default stays off because reasoning is the largest and most private content in a message, and providers often surface a summary rather than the real trace. - **By default the stream carries no reasoning at all.** Only `type: "text"` content parts can enter it, which is a project rule rather than a default that can drift: a dedicated test fails if a `thinking` part ever reaches the file while the switch above is off. - **What the window shows** is smaller than what is stored. A tool block streams at most 3 lines at a time, and its final record shows the last `contentWindowLines` lines (default 10) of the payload. Narration is coalesced before it is ever written, because a live model sends fragments far too small to be worth a record: one is written when a fragment ends a line or a sentence and at least ~80 bytes have accumulated, an unbroken run is released at a 600-byte ceiling cut at the last whitespace rather than inside a word, and anything still held is flushed when the message closes. Measured on real runs: 129 records for 2.2 KB before coalescing, and 28 records for 4.2 KB (150 characters each) after it, without losing a character. Hidden lines and omitted bytes are announced rather than silently dropped, and the header names the **line number** in the stream file that holds the record, so the whole thing is one jump away: ```bash sed -n '137p' ~/AppData/Local/ExpertCouncil/observability/exec_abc123.jsonl | jq -r .text ``` - **Three ceilings, all of them content-only**: `contentEventBytes` bounds one payload (a larger one is stored as head + tail with the gap counted in bytes), `contentFileBytes` bounds one delegation's stream (default 10 MB), and `contentTotalBytes` bounds the whole directory (default 200 MB, evicted oldest-first). Hitting one drops *content* and writes a `stream_truncated` notice; it never drops how the delegation ended, and it can never fail the delegation. An in-flight stream always has the newest modification time, so directory eviction cannot pull the file your window is following. - Files are still pruned after 7 days, and recording adds nothing to what the Main Agent receives: expert results stay the same compact structured summary. ### The observer window's layout An observer window reads the way a coding agent's own transcript does: narration as plain prose with no per-line attribution, and each tool call as a shaded block whose header names the tool and what it was asked to run. ```bash expert-council watch --exec exec_abc123 # panel in a terminal, plain when piped expert-council watch --exec exec_abc123 --style plain # one line per event, as before expert-council watch --exec exec_abc123 --style panel --columns 100 --no-color ``` - **`--style auto` (the default) picks panel for a terminal and plain for anything else**, so redirecting a stream to a file keeps producing exactly the bytes it produced before the panel existed. `--json` neither renders nor colours. - **Blocks are separated from each other, not only from prose.** Two grey bands touching read as a single block, so a blank line now goes between consecutive blocks and the header band is one shade lighter than its body, so a seam survives even when lines do touch. A tool that repaints while it runs is deliberately *not* broken apart: that is one block growing. - **An observer window is told which layout to use.** The launcher passes `--style panel --color` explicitly rather than letting the follower sniff for a terminal: the window it spawns inherits ignored stdio handles, where a sniff reports a pipe, and an operator would otherwise get the plain form inside a real console. `--style auto` stays the right default for a human at their own terminal. - **A block needs a dial that records tool output.** At `none` and `assistant` there is nothing to put inside one, so each call stays a single dim line naming the tool - which is what those dials are for: activity without content. From `assistant+tool-tail` up, the block carries the result. - **The `$ ` header needs arguments to be visible**: `redactToolArgs: false` supplies a bounded one-line summary, and only `transcript+args` stores the full text. With redaction on, the header says `bash` and nothing more - a privacy choice, not a rendering limitation. - **Colour belongs to the view, not the data.** ANSI is emitted only for a terminal, and never when `NO_COLOR` is set or `TERM=dumb`; `--color` / `--no-color` override that. The stream file stays plain JSON either way. Padding a block to the right edge measures East Asian text as two cells per character, so a Chinese narration line does not leave the shading half a cell short. ### Struggle detection (guardrails) `security.guardrails` decides whether the council notices an expert that is stuck rather than merely busy, and what it does about it. Detection is **non-blocking by construction**: it warns, and never aborts. A false positive costs one wasted look; an auto-abort destroys good work. | Option | Default | Effect | | --- | --- | --- | | `warnHost` | `true` | Count struggles and surface them as `attention` in `expert_status({ view: "running" })`, plus one native Pi notice per warning (`expert-council-guardrail`). Off stops the warnings; the counters are still recorded. | | `nudgeExpert` | `true` | Steer the expert itself once per warning: do not repeat an identical failing call; either state the change of approach, `report_and_stop`, or `request_decision`. Hard cap **two nudges per execution**, and only where the runtime session supports steering. | | `consecutiveToolFailures` | `3` | Failing tool calls in a row that raise `consecutive_tool_failures`. | | `minCallsForRatio` / `failureRatio` | `8` / `0.5` | After at least 8 observed calls, a failure fraction at or above 50% raises `failure_ratio_high`. | | `budgetFractions` | `[0.6, 0.85]` | `budget_fraction` warnings at those fractions of the current attempt's `timeoutMs`; only the highest one also nudges. | | `maxTotalWallMs` | unset | Aggregate ceiling across **all attempts of one delegation**. Unset keeps the historical behaviour; when set, the retry loop stops early with an explanatory risk instead of silently spending `retry.maxAttempts x timeoutMs`. | Every warning is also written to the observability stream as an `attention` event, so `expert-council watch` shows it in the operator's terminal. The numbers come from what the runtime observed on its own tool events, not from the expert's account of its run: a delegation result carries `executionMetadata.toolCalls`, `.toolErrors` and `.attention`, plus `executionMetadata.attemptHistory` - one bounded entry per attempt (model, status, failure type, duration, first 300 characters of the attempt summary) - so a host can see why three attempts cost an hour without opening a single state file. A supplier outage says nothing about a model. Transport failures (`Connection error.`, `fetch failed`, `502/503/504`, `gateway timeout`, `overloaded`) classify as `provider_error` instead of `unknown`, and outcomes with that failure type are excluded from the reliability aggregates that feed routing, so an outage cannot make a capable model look unreliable. ### Provider limits & concurrency `route-policy.json` can carry an optional `providers` map next to the model allow/deny policy — see [Route policy file](#route-policy-file) for the allow/deny syntax, which this section does not repeat: ```json { "version": 1, "providers": { "qwen-token-plan-cn": { "maxConcurrency": 2, "dailyTokenCap": 5000000, "weeklyTokenCap": 40000000 }, "zai": { "maxConcurrency": 0 } } } ``` - `maxConcurrency` (integer ≥ 0) caps simultaneous running executions for the provider; `0` or omitted means unlimited. - `dailyTokenCap` and `weeklyTokenCap` (integers > 0) cap weighted token consumption per UTC calendar day and per ISO week (Monday start, UTC). Defaults are `20,000,000` daily and `150,000,000` weekly. - Accounting is weighted: each completed attempt consumes `(inputTokens + outputTokens) × costMultiplier` for the attempt's model; cache read/write tokens are not counted. - Breaching a cap marks every model of the provider in the persisted assessment until the next UTC reset boundary — next UTC midnight for a daily breach, next Monday `00:00` UTC for a weekly breach — so routing stops burning attempts on the depleted provider and retries it automatically after the reset. - In-flight counting uses running executions assigned to the provider; a provider at `maxConcurrency` is excluded from new candidates until one finishes. - When no usage ledger is wired, caps and concurrency limits are disabled and Core performs no I/O. `expert_inspect` reports per-provider `providerLimits` with `maxConcurrency`, both caps, weighted `usedToday`/`usedWeek`, `remainingDaily`/`remainingWeekly`, and `inFlight`. `config/examples/` provides: - `balanced.example.json`: conservative defaults for zero configuration. - `subscription-heavy.example.json`: prioritize subscription capacity, protect scarce quota. - `metered-quality.example.json`: separate economical from high-quality metered APIs. - `qwen-glm.example.json`: explicitly labeled as a hypothetical user preference, not an objective benchmark. ### Capability profiles Models can receive user scores from 0 to 10 on these dimensions: ```text reasoning planning architecture coding debugging review longContext toolReliability bashReliability autonomousExecution speed ``` Model keys must use the exact `provider/model` returned by `models --json`. Newly discovered models without local profiles receive conservative defaults; stale configuration for unavailable models only produces warnings and never crashes routing. ### Main Agent capability audit and first-council preference Users do not need to maintain per-model usage priorities. After the Main Agent decides a task deserves a council, if this is the first council in a new conversation and the user has not expressed a preference, it should ask once: ```text price first (economy) balance price, time, and success rate (balanced) speed first (speed) ``` The Pi Package records the choice in the current Pi session's hidden extension state; later councils in that conversation reuse it automatically unless the user changes it. `economy` boosts cost weights, `speed` boosts the audited speed dimension, and `balanced` uses normal role weights. The legacy `quality` API value remains compatible but is not offered as a default prompt option. Model capabilities are audited by the Main Agent rather than hand-ranked by users. `expert_inspect` returns a mandatory assessment gate: if there is no audit yet, the audit is older than 30 days, the callable model set changed, or the user explicitly requests a re-audit, the Main Agent must research every callable model listed by the gate using the host's own web tools; `expert_build` will not assemble a council until then. The Main Agent submits a complete `modelAssessment` whose ISO timestamp must come from the host's real clock, whose sources are consolidated into 1–12 URLs, and which contains 0–10 capability dimensions plus verifiable provider access/billing evidence. Future-dated timestamps are reported separately and can be corrected without repeating the web research. If the gate reports the saved assessment is still `current`, the host should omit `modelAssessment` when calling `expert_build`; incomplete, stale, or future-dated replacement snapshots cannot displace a current one. The assessment is stored once per user data directory and reused across conversations and workspaces as long as the callable model set remains compatible; ordinary plan and execution state saves never overwrite the global assessment from a stale in-memory snapshot, and only an explicitly submitted assessment that passes the gate replaces the stored scores. Explicit user billing configuration always outranks the Main Agent's judgment; unverifiable billing stays `unknown`. Cross-check rather than trust a single leaderboard: the [Artificial Analysis Data API](https://artificialanalysis.ai/data-api/docs) provides coding, agentic, price, throughput, and latency data; [LiveBench](https://livebench.ai/) covers coding and agentic coding; [Arena](https://arena.ai/leaderboard/text) reflects human preference; provider documentation verifies versions, context, tools, and access method. [OpenRouter Rankings](https://openrouter.ai/rankings?category=programming) mostly reflect real-world usage and are only an adoption signal, not proof of model quality. When the gate demands an audit and the host has no web tools, the Main Agent must state the limitation and stop instead of silently using unaudited defaults; Expert Council never auto-installs plugins, Skills, or third-party executables. ### Role weights Every semantic role has normalized default weights. Implementation Worker emphasizes tool reliability, coding, autonomous execution, and shell reliability; Architecture Oracle emphasizes architecture, planning, long context, and review. ```json { "routing": { "roleWeights": { "implementation-worker": { "toolReliability": 0.4, "coding": 0.3, "costEfficiency": 0.1 } } } } ``` Weights are re-normalized automatically. `costPolicy: economy` boosts the cost factor, `speed` boosts speed and reduces cost, and the legacy `quality` reduces cost; none of them bypass safety or compatibility hard constraints. ## Semantic roles and team sizing Roles are defined by task semantics and never bound to a model name: | Role | Default permissions | Purpose | |---|---|---| | Planner | read-only | Task decomposition, dependencies, and risks | | Scout | read-only | Repository exploration and context compression | | Architecture Oracle | read-only | Hard cross-file reasoning and second opinions | | Implementation Worker | writable | Bounded code changes and focused testing | | Debugger | writable | Reproduce, isolate, fix, and verify | | Reviewer | read-only | Regressions, edge cases, and design review | | Verifier | read-only | Check reported tests, diffs, and acceptance criteria | Tiny tasks use one Worker; normal tasks use Worker plus Verifier; complex features use Planner, Worker, Reviewer, and Verifier; complex debugging uses Scout, Debugger, Oracle, and Verifier. `maxExperts` caps team size, and the Main Agent is never duplicated as a redundant `lead` expert. Read-only experts run in the main workspace and therefore **never report `filesChanged`**: they have no mutation tool, so anything git-dirty there belongs to the Main Agent and attributing it to the expert would fabricate authorship. `expert_cleanup` on such a run answers `not-required` because no worktree exists. If you need changes made, dispatch an implementation-worker or debugger and review its isolated worktree instead. A writable run whose diff *could not* be read is the other case, and it is reported as such: `executionMetadata.filesChangedError` plus a matching risk, because an empty list produced by a failing `git status` would otherwise read as "the expert changed nothing" and a host integrating from that list would throw real work away (defect #28). Task classification and all scoring math are deterministic. Hosts can inspect the selected model, alternatives, scores, and concise reasons before delegating. Council assembly also applies configurable diversity penalties; Reviewers prefer a different provider and inferred model family from earlier members when economical, but role fit and hard constraints still dominate. ## Routing pipeline ```text discover currently callable candidate models -> apply hard constraints -> merge capability and profile layers -> score role fit and effective cost -> conservatively apply local outcome adjustments -> rank with stable tie-breaking -> return selection, alternatives, and reasons ``` Hard constraints reject unavailable or disabled models, incompatible roles, insufficient tool reliability, insufficient context, mutation without runtime support, `escalation-only` resources on routine tasks, and models carrying an active runtime availability marker. Pi caches its model inventory per session and provider catalogs can keep stale model names, so a council could otherwise be assembled around a model the upstream can no longer serve. When a delegated attempt fails as `provider_error` with dead-model evidence (for example `model_not_found`, unknown or discontinued models, or the runtime's own availability pre-check), the service records a `modelAvailability` marker into the persisted shared model assessment (`EXPERT_COUNCIL_DATA_DIR`, i.e. `%LOCALAPPDATA%/ExpertCouncil/model-assessment.json` on Windows) through an atomic read-modify-write that never reverts a newer snapshot written by another running Pi/Codex instance. The affected `expert_result` names the model in `executionMetadata.unavailableModels` and `risks`, `expert_inspect` warns about active markers, and later `expert_build`, delegation, and escalation hard-reject marked models. Markers are conservative local evidence: they expire after 24 hours, survive freshly submitted audits, and an explicit `modelOverrides["provider/model"].overrideUnavailableMarker: true` re-enables a model. Markers are not limited to dead models: a transient TPM/RPM rate limit records a `rate-limited` marker for `MODEL_RATE_LIMIT_MARKER_TTL_MS` (2 minutes) and also covers the same provider's sibling models, because throttling is an account-level condition; an unreachable provider transport records a `transport-unstable` marker for 5 minutes against the failing route only; plan or balance exhaustion records `quota-exhausted` (6 hours); and a plan-level access denial - a 403 `AccessDenied.Unpurchased`, for instance - is dead-model evidence and records `unavailable` (24 hours). None of the transient kinds counts against the model's own reliability record: that data belongs to the model, this data belongs to the supplier. When no saved assessment exists yet, the marker cannot be persisted, but the failure is still reported to the Main Agent and recorded in local telemetry. Reasoning levels are optional and model-specific. A configured role preference applies only when Pi exposes it for the selected model; otherwise Pi keeps or clamps to the model's supported default. ## Skills and least privilege The canonical platform-neutral host guidance is [`shared/skills/expert-council/SKILL.md`](https://github.com/Labiey/expert-council-router/blob/main/shared/skills/expert-council/SKILL.md). The build composes it with the small host overlays under `shared/skills/expert-council/hosts/` into separate Pi and Codex `SKILL.md` artifacts without duplicating the shared workflow. The Pi artifact only describes completion `steer`/`followUp` behavior and never exposes `expert_wait`; only the Codex artifact describes the bounded `expert_wait` flow. Shared role prompts live in `packages/core/src/roles/prompts/` and are copied as package assets rather than rewritten per host. Read-only roles never receive `edit`, `write`, `bash`, or `powershell`, even if a caller tries to include them. Pi sessions use a real `tools` allowlist, which is stronger than prompt-only guidance. Until a dedicated non-mutating command runner exists, tasks that need shell-driven testing belong to writable roles in isolated worktrees. Only installed, enabled Pi Skills required by the role are activated. User-scope Skills are trusted by default; project and temporary Skills are excluded unless their exact name is listed in `security.trustedSkills`. Every expert resource loader disables extensions, prompt templates, themes, and project context files, and Pi SDK versions that cannot enforce these policies are rejected. Expert Council never downloads or installs Skills or executable extensions. Expert prompts require: read before editing, verify paths, prefer targeted edits, diagnose failures before changing approach, use finite non-interactive commands, inspect results, never delegate recursively, and return compact JSON instead of private reasoning. ## Retry and escalation Failure types are normalized to: ```text tool_call_error reasoning_failure test_failure timeout provider_error missing_context permission_error unknown ``` By default, the first correctable failure - a tool-call error or a test failure - receives at most one retry with a changed approach. A missing-context failure, a permission or workspace denial, or an expert-declared block ends the delegation immediately instead, because a different model would hit the same wall. Repeated relevant failures or provider errors switch to the next eligible, untried model. Attempt and escalation budgets are separately capped; with no candidates or exhausted budget, an unresolved state returns to the Main Agent. There are no infinite loops, and the same failed action is never repeated blindly by policy. ## Structured results and context efficiency Expert results contain status, role, model, summary, changed files, tests, findings, risks, recommended next action, failure type, the approximate usage Pi exposes, and bounded execution metadata. The system prefers the expert's own structured failure type and deterministically classifies test, provider, tool, and context failures; unparseable non-JSON output is marked `reasoning_failure`. Private reasoning is never requested or stored, and whole source files are never copied back into the Main Agent's context. ## Workspace safety The system never assumes that Codex's own sandbox contains the external Pi process. The Pi Runtime uses an independent boundary: 1. Canonicalize the requested workspace path. 2. Require the path to fall under allowed roots. 3. Create a detached worktree from the repository's current `HEAD` inside a current-user-private directory under the system temp directory. 4. Give the Worker write tools inside that worktree. 5. Return the worktree path and changed-file list. 6. The Codex or Pi Main Agent inspects, integrates, and finally accepts. 7. After accepting or rejecting, call `expert_cleanup`. One call removes every worktree created for that execution ID through retries or escalations and returns all removed paths. Unclaimed worktrees are pruned automatically after `security.worktreeRetentionMs` (24 hours by default), and Git metadata is pruned alongside. Non-Git workspaces refuse writes by default. If in-place mutation is truly required, it must be explicitly enabled: ```json { "security": { "workspaceStrategy": "bounded-in-place", "allowInPlaceMutations": true, "allowedWorkspaceRoots": ["/absolute/path/to/project"] } } ``` Read [`SECURITY.md`](https://github.com/Labiey/expert-council-router/blob/main/SECURITY.md) before enabling. ## Telemetry and local learning The default user data root is `%LOCALAPPDATA%\ExpertCouncil` on Windows, `$XDG_STATE_HOME/expert-council` or `~/.local/state/expert-council` on Linux, and `~/Library/Application Support/ExpertCouncil` on macOS. The shared `telemetry.jsonl` stores opaque execution outcomes so real reliability can be reused across conversations and workspaces; feedback for the same execution overwrites earlier samples and never double-counts. The shared `model-assessment.json` stores the latest explicit capability and billing scores plus runtime-learned model availability markers. `EXPERT_COUNCIL_DATA_DIR`, `EXPERT_COUNCIL_TELEMETRY`, `EXPERT_COUNCIL_MODEL_ASSESSMENT`, and `EXPERT_COUNCIL_STATE` override locations. It never records prompts, source content, credentials, API keys, secrets, or reasoning. Aggregate metrics include per-role success rate, first-pass rate, tool error rate, retry rate, verification pass rate, and average attempts. Expert Council has no remote analytics endpoint. Plans, execution state, and completed structured results remain workspace-scoped in `workspaces//state.json`. They survive process restarts; a task still running at restart is closed as an explicit interrupted failure. Legacy in-project `.expert-council` and `%USERPROFILE%\.expert-council` directories are not deleted automatically. ## CLI The CLI uses exactly the same Core and Pi Runtime as MCP and the Pi Package: ```text expert-council models expert-council inspect expert-council compositions expert-council build expert-council delegate expert-council feedback --verification passed|failed expert-council status expert-council abort expert-council reset expert-council verify (--exec | --workspace ) --command JSON_ARRAY expert-council respond --kind decision|tool_approval expert-council cleanup expert-council watch --exec [--follow] expert-council --version ``` Common flags: - `--json`: machine-readable output. - `--cwd`: project workspace. - `--config`: user policy file. - `--telemetry`: custom local telemetry path. - `--state`: custom path for persisted plans, executions, and results. - `--timeout-ms`: expert execution timeout. - `watch` also takes `--dir` (event stream directory), `--interval-ms` (poll interval, default 1000), and `--timeout-ms` (maximum follow time, default 300000), and `--quiet-ms` (how long a stream with no final marker must stay silent before the follower gives up, default 15000). It needs `security.observability.expertWindow: "interactive"` to be producing a stream. - `watch` also takes `--max-lines N` (body lines per recorded content block, 1-50, default 10) and `--max-chars N` (clamp per shown body line, 20-400, default 120); an auto-opened window inherits `contentWindowLines` and `contentWindowChars` from configuration, so raising the configured number is not silently overridden by the follower's default. ## MCP Server The MCP surface is deliberately limited to thirteen semantic tools: - `expert_inspect` - `expert_build` - `expert_delegate` - `expert_wait` - `expert_result` - `expert_abort` - `expert_feedback` - `expert_cleanup` - `expert_escalate` - `expert_status` - `expert_availability_reset` - `expert_verify` - `expert_respond` `expert_respond` answers a running expert's open `pendingInteraction` (a `request_decision` choice or a `request_tool` grant) so its session continues; a headless host discovers open interactions by polling `expert_status` with `view: "running"` or `expert_result` with `includeProgress`. `expert_inspect` and `expert_build` return compact host-facing views by default. Pass `detail: "full"` only when exact model metadata, alternatives, scores, tools, or Skills are genuinely required. ### Council compositions Saved council rosters live in `council-compositions.json` next to `route-policy.json` in the shared state directory — no dedicated tool. The default location is `%LOCALAPPDATA%\ExpertCouncil\council-compositions.json` on Windows, `~/Library/Application Support/ExpertCouncil/council-compositions.json` on macOS, and `$XDG_STATE_HOME/expert-council/council-compositions.json` (or `~/.local/state/expert-council/council-compositions.json`) on Linux; override it with `EXPERT_COUNCIL_COMPOSITIONS`. ```json { "compositions": [ { "name": "daily-cheap", "roles": { "scout": ["qwen-token-plan-cn/deepseek-v4-flash"], "implementation-worker": ["zai/glm-5.3-flash", "deepseek/deepseek-v4-flash"] } } ], "sessions": { "01a066b2-a166-7e67-983d-2bbec848c223": "daily-cheap" } } ``` - `compositions` is an ordered list (menu priority) of at most 32 unique names (≤80 characters). `roles` is keyed by the seven semantic role values; each value is a list of `provider/id` model keys (≤16 per role). A missing or empty role auto-routes that role. - `sessions` maps a session key (host conversation id) to a composition name. Entries expire after 30 days; bindings written by the tool carry an `updatedAt` stamp, while hand-written string entries are kept. - On the first build with neither `composition` nor `costPolicy`, `expert_build` returns a `compositionMenu`: up to three saved rosters plus an `auto` option. Pass a saved name back as `composition`; the `auto` option is the cost-policy flow (economy/balanced/speed). - A successful explicit `composition` build binds that name to the session; a successful `costPolicy` build unbinds it. A menu response binds nothing. - A role's candidate pool is the composition list intersected with route-policy filtering and provider cap/concurrency exclusions. Route-policy `deny` always wins over a pool, and a fully excluded role is reported as unstaffable. - `expert_delegate` accepts an optional `model` (`provider/id`) per assignment and in `assignments[]`. Pin different pool models on same-role assignments to dispatch several experts concurrently; a pin outside the role's pool, the discovered inventory, or the route policy fails with a structured error. ### Route policy file Model allow/deny lists live in `route-policy.json` next to `model-assessment.json` in the shared state directory — no dedicated tool. The file holds a `system` entry that every session obeys and per-session entries under `sessions` keyed by the host conversation (Pi session IDs survive resume; MCP stdio conversations use a stable `"default"` key). Sessions may only narrow the system policy: deny lists union, allow lists intersect, and deny always wins. Entries are `provider/id` or a bare `provider` for a whole provider. `expert_inspect` returns this conversation's `sessionKey`, the `effective` policy, and the file's `sourcePath` so the host (or you) can edit it directly; changes apply on the next expert call, stale session entries are pruned after 30 days, and a corrupt file is ignored with a warning. ```json { "version": 1, "system": { "allow": [], "deny": ["acme-host", "acme-metered/acme-pro"], "updatedAt": "2026-09-18T12:00:00+08:00", "note": "free text, never sent to an expert" }, "sessions": { "4f0c…": { "allow": ["acme-plan/acme-flash"], "deny": ["acme-plan/acme-heavy"], "updatedAt": "2026-09-18T12:00:00+08:00", "note": "narrows the system policy for this conversation only" }, "default": { "deny": ["acme-metered"] } }, "providers": { "acme-plan": { "maxConcurrency": 1, "dailyTokenCap": 12000000, "weeklyTokenCap": 60000000 }, "acme-metered": { "maxConcurrency": 2 } } } ``` - `version` must be the literal `1`; anything else rejects the document. - `system` applies to every session. `sessions` is keyed by host conversation id - Pi session ids survive a resume, and MCP stdio conversations share the stable `"default"` key. - A session may only **narrow** the system policy: deny lists are unioned, allow lists are intersected, and deny always wins. An `allow` list is exclusive - naming models there hides every model not listed. - Entries are `provider/id` or a bare `provider` to cover a whole provider. At most 32 entries per list, 200 characters each; control characters are stripped and anything that is not `provider` or `provider/id` is dropped rather than trusted. - `providers` holds spending and concurrency limits, **not** allow/deny: `maxConcurrency` (0-8 per provider, `0` parks it), `dailyTokenCap` and `weeklyTokenCap`, all positive integers. Caps are enforced against the usage ledger, and a provider that hits its cap is skipped for the window rather than failing the delegation. - `note`, `updatedAt` and `workspace` are recorded metadata. `workspace` in particular is stored and round-tripped but no routing code consults it yet - do not rely on it to scope a policy. - Deleting the file is not neutral: with no deny list, every previously excluded provider becomes routable again, which can silently move work from a subscription plan onto metered APIs. - `expert_inspect` reports this conversation's `sessionKey`, the `effective` policy and the file's `sourcePath`, so the host (or you) can edit it directly; changes apply on the next call, stale session entries are pruned after 30 days, and a corrupt file is ignored with a warning. ### Delegating several experts at once `expert_delegate` starts background work and immediately returns execution IDs; the original single-assignment parameters remain compatible. `timeoutMs` is required for every assignment (1000–3600000 ms) — omitting it is an error; set it from expected task difficulty. A timed-out attempt scales the retry budget by 1.5× automatically, and experts stop early with a structured `missing_context`/`permission_error` result when a task is impossible with their assigned tools. When two or more independent tasks exist, dispatch the entire batch before continuing other Main Agent work: ```json { "assignments": [ { "role": "scout", "task": "locate relevant files", "taskDescription": "repo mapping", "timeoutMs": 300000 }, { "role": "reviewer", "task": "review boundary design", "taskDescription": "boundary review", "timeoutMs": 600000 } ] } ``` `assignments` must be an actual JSON array, never a string containing JSON. The native Pi adapter offers bounded compatibility parsing for models that occasionally stringify the array, but normal callers should emit real arrays. The optional `taskDescription` is a short host-facing label for identifying the task; it is not part of the expert's actual task content. After dispatching, the Main Agent should continue all independently completable work; when nothing useful remains, call `expert_wait` once with up to eight execution IDs, usually `mode: "all"` (use `"any"` when any early result unblocks progress), and a `timeoutMs` sized to the estimated remaining difficulty. Waiting is driven by execution-promise completion events rather than polling; blocking the current MCP call is expected behavior, and no main-model tokens are consumed while waiting. ### Waiting for background work ```json { "executionIds": ["exec_a", "exec_b"], "mode": "all", "timeoutMs": 900000 } ``` `expert_wait` returns only completion state and task IDs; fetch the formal feedback with `expert_result` and call `expert_feedback` after the Main Agent's acceptance. `expert_wait.timeoutMs` bounds only that wait and never extends each expert's own execution deadline. Every potentially blocking Expert Council, Bash, PowerShell, or other MCP call must still carry an explicit finite timeout sized to the operation; remaining synchronous Expert Council operations are protected by an independent 30-second in-server cap, with one deliberate exception: `expert_verify` is bounded by the larger of that cap and 60 seconds, because turning an expert claim into an observed exit code usually means running a build or a test suite. `expert_status` returns a bounded per-attempt history. The native Pi Package uses proactive completion notifications and therefore does not expose `expert_wait`. ### Running the stdio server directly Start the stdio server directly: ```bash node packages/mcp-server/dist/bin.js ``` ### Environment variables Supported environment variables: - `EXPERT_COUNCIL_WORKSPACE`: default allowed workspace. - `EXPERT_COUNCIL_CONFIG`: user JSON configuration. - `EXPERT_COUNCIL_TELEMETRY`: local telemetry JSONL path. - `EXPERT_COUNCIL_STATE`: persisted plans, executions, and results state path. - `EXPERT_COUNCIL_WORKTREES`: parent directory for private expert mutation worktrees (the per-user private subdirectory is always kept); useful for short-path volumes or a faster disk. - `EXPERT_COUNCIL_DATA_DIR`: root for all per-user council data - assessments, route policy, usage ledger, recovery state, and observability streams; the platform default is `%LOCALAPPDATA%/ExpertCouncil` on Windows. - `EXPERT_COUNCIL_CONTENT`: content dial for this process only, overriding `contentStream` and `contentByRole`; an unrecognised value is reported as a limitation rather than guessed at. - `EXPERT_COUNCIL_MODEL_ASSESSMENT`: persisted shared model assessment path. - `EXPERT_COUNCIL_ROUTE_POLICY`: persisted per-session routing allow/deny policy path. - `EXPERT_COUNCIL_USAGE_LEDGER`: persisted token usage ledger behind provider caps. - `EXPERT_COUNCIL_COMPOSITIONS`: saved council compositions path. - `EXPERT_COUNCIL_MCP_TIMEOUT_MS`: bounded timeout for synchronous MCP operations, 30000 ms by default. - `PI_CODING_AGENT_MODULE`: explicit Pi package directory when automatic resolution fails. Environment overrides and CLI path flags are trusted operator inputs. In particular, `PI_CODING_AGENT_MODULE` loads executable code, while config, workspace, telemetry, and state paths select local files; never accept them from an untrusted repository, task text, or model output. ## Native Pi Package For daily use, install from npm (see [Quick start](#install-the-pi-package)); this section covers source development and local candidate validation. Build and install the local candidate from the repository root. Even on Windows, use forward slashes whenever a command may pass through Pi's Bash-compatible shell; an unquoted `.\packages\pi-package` loses its backslashes before reaching Pi. ```bash npm run build pi install "./packages/pi-package" pi list pi --verbose ``` `pi list` should show the configured source and its resolved absolute package directory. A newly started verbose Pi session should list `dist/extension.js`, the `expert-council` Skill, and the twelve semantic tools without `expert_wait`. Running Pi processes do not hot-reload a rebuilt or removed package. Or load it for one run without persisting: ```bash pi --verbose -e "./packages/pi-package" ``` A zero-cost load check simply asks Pi to call `expert_inspect`. A real orchestration check should, in a new conversation, build one read-only council, batch two independent read-only assignments at once, confirm that `expert_delegate` immediately returns execution IDs, then accept each completion with `expert_result` and `expert_feedback`. When testing a writable expert, also confirm that one `expert_cleanup` call reports the worktree for the execution in `workspaces` (retries reuse and reset it instead of creating another) and that `git worktree list` afterwards contains only the main checkout. Before removing a persistent installation, exit every Pi process that loaded the package, then run from the same repository root: ```bash pi remove "./packages/pi-package" pi list ``` If the working directory changed, pass the resolved absolute path instead. PowerShell example: ```powershell $ecPiPackage = (Resolve-Path "./packages/pi-package").Path pi remove "$ecPiPackage" pi list ``` If removal runs through Pi's Bash-compatible shell, use the forward-slash absolute path printed by `pi list`, for example `pi remove "C:/path/to/ExpertCouncil/packages/pi-package"`. Do not copy the indented relative source shown by `pi list` unless the command is resolved from the same settings-directory context. Pi loads `dist/extension.js` and the synchronized `expert-council` Skill through the package manifest's `pi.extensions` and `pi.skills`. The extension registers the twelve semantic tools; because native Pi already provides completion `steer`/`followUp`, the MCP-only `expert_wait` is omitted. It contains no second routing implementation. Pi delegation is non-blocking; a single call can start up to eight independent background assignments before returning. When an expert finishes, the extension sends compact JSON containing the completed `executionId` and, only when supplied at dispatch, the `taskDescription`; it never carries feedback directly. Notifications use `steer` while the Main Agent is working and a `triggerTurn` `followUp` when it is idle. The Main Agent then calls `expert_result` for the structured feedback. Dispatch the entire ready batch before ending the turn, and avoid polling or silently waiting afterwards. ## Codex plugin The Codex plugin turns Codex into the Main Agent: it bundles the shared `expert-council` Skill and a stdio MCP Server, while experts execute through Pi. It ships no hooks — the server uses MCP client roots when available, otherwise derives the task workspace from Codex's host-owned `codex/sandbox-state-meta` capability, then falls back to the `EXPERT_COUNCIL_WORKSPACE` override, and refuses to use the plugin installation directory as a workspace. The built plugin root is: ```text packages/codex-integration/plugin/expert-council/ .codex-plugin/plugin.json .mcp.json skills/expert-council/SKILL.md dist/server.mjs dist/roles/*.md THIRD_PARTY_NOTICES.md ``` ### Installation The current release ships both the prebuilt MCP server and its tested Pi SDK runtime, so Codex can install the plugin directly from the repository as a pinned Git marketplace. Node.js 22.19 or newer and an already configured Pi account/model catalog are required; cloning this repository, running `npm install`, or resolving a global `@earendil-works/pi-coding-agent` module is not required. ```bash codex plugin marketplace add Labiey/expert-council-router --ref v0.8.6 --json codex plugin marketplace list --json codex plugin list --marketplace expert-council-router --available --json codex plugin add expert-council@expert-council-router --json codex plugin list --json ``` `marketplace add` is needed only once for this release. If the marketplace name is already registered from an older or local source, remove that source first or follow the upgrade procedure below. `plugin list --json` should show `expert-council` as installed from `expert-council-router`. Codex Desktop on Windows includes the CLI, but it may not be on `PATH`. In PowerShell, resolve the running Desktop binary and use it for the same remote installation: ```powershell $ecCodex = (Get-Command codex.exe -ErrorAction SilentlyContinue).Source if (-not $ecCodex) { $ecCodex = Get-Process codex -ErrorAction SilentlyContinue | Where-Object Path | Select-Object -First 1 -ExpandProperty Path } if (-not $ecCodex) { $ecCodex = Get-ChildItem (Join-Path $env:LOCALAPPDATA "OpenAI/Codex/bin") ` -Filter codex.exe -File -Recurse -ErrorAction SilentlyContinue | Sort-Object LastWriteTime -Descending | Select-Object -First 1 -ExpandProperty FullName } if (-not $ecCodex) { throw "Codex Desktop CLI was not found." } & $ecCodex plugin marketplace add Labiey/expert-council-router --ref v0.8.6 --json & $ecCodex plugin marketplace list --json & $ecCodex plugin list --marketplace expert-council-router --available --json & $ecCodex plugin add "expert-council@expert-council-router" --json & $ecCodex plugin list --json ``` Fully quit Codex Desktop, wait for its backend process to exit, reopen it, and start a new task. Merely opening another task is not a reliable MCP reload boundary in every Desktop build. For local plugin development, clone the repository, run `npm ci && npm run build`, and pass its absolute root to `codex plugin marketplace add` instead of the GitHub repository name. The pinned remote release is recommended for normal use. A correct load exposes the `expert-council` Skill and all thirteen `expert_*` MCP tools. `expert_inspect` must return a real inventory rather than a "No compatible Pi SDK is installed" diagnostic. If the Skill is present but the tools are absent, or inspection reports that diagnostic, verify that the marketplace is pinned to `v0.8.6` or newer, then restart or reinstall the plugin instead of launching `dist/server.mjs` manually or sending hand-written JSON-RPC. To verify the installed workflow, use a new Codex task and ask: ```text Use Expert Council to inspect the currently available Pi models, providers, billing classification, and model-assessment status. Return only a compact summary; do not build a council or delegate experts. ``` The task should invoke `expert_inspect`. It should not ask for hook trust, manually launch the MCP server, or treat the plugin cache as the project workspace. ### Upgrade A marketplace pinned with `--ref` intentionally stays on that release. To upgrade, remove the installed plugin and old marketplace registration, then add the new tag and reinstall: ```bash codex plugin remove expert-council@expert-council-router --json codex plugin marketplace remove expert-council-router --json codex plugin marketplace add Labiey/expert-council-router --ref vX.Y.Z --json codex plugin add expert-council@expert-council-router --json ``` Replace `vX.Y.Z` with the intended release. Users who deliberately track the default branch can omit `--ref` and later run `codex plugin marketplace upgrade expert-council-router --json`, but pinned tags are safer for normal use. After reinstalling, fully restart Codex Desktop and test in a new task. Avoid installing multiple copies that all declare the `expert_council` MCP server. ### Behavior highlights - The bundled `.mcp.json` raises the host tool-call ceiling to 3660 seconds so a single bounded `expert_wait` can block until completion; the Skill still requires explicit per-operation deadlines rather than treating that ceiling as a default budget. - On the first build of a conversation with neither `composition` nor `costPolicy`, `expert_build` returns a composition menu (up to three saved rosters plus an `auto` option); choosing `auto` establishes the cost policy (economy, balanced, or speed) for the session. Until a menu choice is made, `expert_build` responses keep asking. - Writable experts mutate inside a detached Git worktree under the trusted workspace; changes come back for Main Agent review and are never auto-merged. ### Removal Remove the plugin and its marketplace registration: ```bash codex plugin remove expert-council@expert-council-router --json codex plugin marketplace remove expert-council-router --json codex plugin list --json codex plugin marketplace list --json ``` If PowerShell cannot find `codex`, locate the Codex Desktop CLI first (while Codex Desktop is running its path can be taken from the process; otherwise fall back to the install directory): ```powershell $ecCodex = Get-Process codex -ErrorAction SilentlyContinue | Where-Object Path | Select-Object -First 1 -ExpandProperty Path if (-not $ecCodex) { $ecCodex = Get-ChildItem (Join-Path $env:LOCALAPPDATA "OpenAI\Codex") ` -Filter codex.exe -File -Recurse -ErrorAction SilentlyContinue | Sort-Object LastWriteTime -Descending | Select-Object -First 1 -ExpandProperty FullName } if (-not $ecCodex) { throw "Codex Desktop's codex.exe was not found." } ``` Then uninstall through the located CLI: ```powershell & $ecCodex plugin remove "expert-council@expert-council-router" --json & $ecCodex plugin marketplace remove "expert-council-router" --json & $ecCodex plugin list --json & $ecCodex plugin marketplace list --json ``` If the plugin still shows after reopening Codex Desktop, close Codex and safely remove the stale Expert Council caches (only `%USERPROFILE%\.codex\plugins\cache\expert-council-*` is touched): ```powershell $ecCacheRoot = [IO.Path]::GetFullPath( (Join-Path $env:USERPROFILE ".codex\plugins\cache") ) $ecCachePrefix = $ecCacheRoot.TrimEnd("\") + "\" $ecTargets = Get-ChildItem -LiteralPath $ecCacheRoot ` -Directory -ErrorAction SilentlyContinue | Where-Object Name -Like "expert-council-*" foreach ($ecTarget in $ecTargets) { $ecResolved = [IO.Path]::GetFullPath($ecTarget.FullName) if ( $ecResolved.StartsWith( $ecCachePrefix, [StringComparison]::OrdinalIgnoreCase ) -and (Split-Path $ecResolved -Leaf) -like "expert-council-*" ) { Write-Host "Removing cache: $ecResolved" Remove-Item -LiteralPath $ecResolved -Recurse -Force } } ``` Fully quit Codex Desktop before starting new tasks. ## Testing ```bash npm test npm run typecheck npm run build npm run pack:check npm run validate ``` `npm run validate` builds first so a fresh clone has the workspace package entries generated before testing. Tests cover model normalization, published prices versus real policy billing, worker reliability, oracle scoring, reviewer diversity, hard constraints, unknown and missing models, team sizing, retries with per-attempt diagnostics, structured failure classification, escalation, retry limits, role permissions, compact host output, configuration validation, telemetry privacy/feedback/usage aggregation, Core host independence, mocked Pi discovery and execution, CLI JSON, MCP schemas, real Pi 0.85.1 extension loading/wrapping and async batch notifications, Pi extension registration, and real Git worktree isolation. Ordinary tests only use the mock runtime and never call paid models. A real read-only Pi execution requires both an explicit model and an explicit cost acknowledgement: ```powershell $env:EXPERT_COUNCIL_LIVE_MODEL = "provider/model" $env:EXPERT_COUNCIL_LIVE_CONFIRM = "YES" npm run smoke:live:pi ``` Ordinary validation flows never execute that script. ## Publishing Run `npm run validate` first, inspect every `npm pack --dry-run` file list, then publish in dependency order: ```text @expert-council/core @expert-council/pi-runtime @expert-council/cli @expert-council/mcp-server @expert-council/pi-package ``` ## Known limitations - Guardrail timing depends on the host process actually being scheduled: a suspended or CPU-starved process can raise a `budget_fraction` warning later than the timeout it was meant to anticipate. - A nudge requires the runtime session to support steering. Pi sessions do; against a runtime that does not, the council warns the host and stays out of the expert's way. - Tool-failure counters record only what the runtime could observe. They are not an audit surface, and a tool that hangs instead of failing contributes no failure count until the attempt itself times out. - Pi's API moves quickly. The current release was verified against the local 0.85.1 SDK (the version pinned in `package.json`); the runtime checks required SDK, model-runtime, resource-loader, and session methods and lists any missing contract explicitly on incompatibility. - Pi has no unified real billing-type API. Runtime subscription signals and named Token Plans take priority; otherwise non-zero catalog prices are treated as metered, and providers without reliable evidence stay `unknown` until an audit or explicit user configuration confirms them. - Expert Council does not infer subjective coding quality from model names, nor does it download benchmark presets automatically. - Detached worktrees start from the committed `HEAD` and do not copy uncommitted changes from the main workspace. This is deliberate isolation; the runtime detects a dirty source workspace and surfaces the deviation through runtime limitations and mutation-council warnings before delegation. - Worktree changes are returned for Main Agent review and are never auto-merged or applied; call `expert_cleanup` after acceptance or rejection, or they will be cleaned up automatically after the retention window. - Writes to non-Git workspaces require explicit in-place mutation authorization. - In-flight model calls do not resume after a server restart; persisted state closes them as explicit interrupted failures while preserving plans and completed results. - The interactive expert window is best-effort observability, not an audit log: event files are bounded, written asynchronously, may drop events under I/O failure, and are pruned after 7 days. They can only be followed from the same data directory as the host process running the experts, so a second checkout or a different `EXPERT_COUNCIL_DATA_DIR` sees nothing. - Codex's own sandbox does not automatically contain the external Pi runtime, so Expert Council uses separate allowed roots and worktree boundaries. - Expert Council provisions mutation worktrees from the repository's own committed lockfile only when `security.workspaceProvisioning.mode` is `auto` (default `none`, so nothing is installed unless opted in). Installs use the lockfile-pinned command with a scrubbed child environment, and with script suppression where the toolchain supports it (`--ignore-scripts` for npm/pnpm/bun, settings environment variables for Yarn); drivers that execute ecosystem build code with no supported opt-out - `uv` builds source distributions, `cargo` runs `build.rs` - are declared as such and surface a limitation, because a lockfile bounds what gets installed, not whether installed code runs; read-only workspaces are never provisioned and ecosystems without a supported lockfile are skipped. After provisioning, a verification gate runs the repository typecheck and test commands, and a failing gate downgrades an otherwise successful result to `partial` with `failureType: "test_failure"` so the corrected-retry path engages. Expert Council still contains no recursive expert trees, graphical interface, remote control plane, or remote telemetry. ## License MIT; see [`LICENSE`](LICENSE).