# Architecture Blueprint: Multi-Agent Smart Agent Extension for Pi (`sagent`) ## 1. Executive Summary & Normative Principles This architecture blueprint specifies an enterprise-grade, self-improving multi-agent orchestration extension for **pi** (`@earendil-works/pi-coding-agent`) named **`sagent`** (short for **Smart Agent**). The extension transforms a standard Pi session into a coordinated, autonomous agent swarm where: - The **Conductor (Product Owner / Task Manager)** orchestrates task dependency graphs (DAGs) using a durable, crash-safe state machine with strict single-writer atomic manifest persistence. - Sub-agents operate in **isolated context windows** with strict least-privilege toolsets and **Git Worktree-isolated** file workspaces with full-range path lease enforcement, post-rebase QA validation, and a strict clean-workspace policy for mutating tasks. - A **Logical Model & Provider Routing Engine** dynamically selects the optimal model per persona by querying Pi's live model registry (`ctx.modelRegistry`), verifying OAuth subscription status (`modelRegistry.isUsingOAuth(model)` and `provider.auth.oauth?.isSubscription === true`), applying conservative billing defaults (defaulting API keys and ambiguous providers to `metered`), and enforcing hard pre-dispatch spend caps. - A **3-tier Knowledge Hierarchy (Project → Profile → Global)** maintains human-readable, credential-sanitized `AGENTS.md` files paired with append-only JSONL event logs. Derived **Vector DBs** (`sqlite-vec`) live in a dedicated **local system cache root** (`~/.cache/sagent/vectors/`) strictly outside cloud synchronization scopes. - **Governed Continuous Self-Improvement**: The agent safely documents itself and refines personas (`.candidate.md`). Code-level self-upgrades execute through a **Candidate-Isolated Upgrade Supervisor** featuring **fail-closed real OS sandboxing**, **supply-chain lockfile controls**, **supervisor-issued one-time approval challenges**, **durable migration checkpoint journals**, **multi-point health verification**, and **out-of-band crash recovery**. Autonomous self-upgrading is **disabled by default**. --- ## 2. System Architecture & Component Diagram ``` ┌────────────────────────────────┐ │ User Prompt / Task │ └───────────────┬────────────────┘ ▼ ┌──────────────────────────────┐ │ Conductor (PO / TM) │◄────────┐ │ - Durable DAG State Engine │ │ │ - Scored Model/Auth Router │ │ │ - Worktree & Lease Manager │ │ └───────┬───────────────┬──────┘ │ ┌─────────────────────┘ └──────────┐ │ ▼ ▼ │ [Sync Critical Path] [Tracked Low-Priority Nodes] ┌─────────────────────────────────┐ ┌────────────────────────────┐ │ 1. Pre-Dispatch Knowledge Query │ │ • Librarian (JSONL Log & │ │ 2. Explorer (Sandboxed fd/rg) │ │ Local Vector Rebuild) │ │ 3. Architect (Deep Thinking) │ │ • Accountant (Self-Docs & │ │ 4. Common Builders (Worktrees) │ │ Changelog Generation) │ │ 5. Special Builder (Gated IO) │ │ • Principal (Self-Evolution│ │ 6. Quality Assurer (QA Gate) ├──(Rework ≤ 2)───────┤ & Upgrades Drafting) │ └────────────────┬────────────────┘ └────────────────────────────┘ │ (QA Pass & Full-Range Lease Check) ▼ [Conductor Worktree Merge] ──► [Clean Working Directory] ``` --- ## 3. Logical Model & Provider Routing Engine ### A. Live Discovery, Pi Auth APIs & Billing Classification The router does not infer billing from configuration presence alone. It queries Pi's official authentication and provider APIs: - **Pi Runtime Model Registry APIs**: - `ctx.modelRegistry.getAvailable()` — Effective models available to active session. - `ctx.modelRegistry.isUsingOAuth(model)` — Determines whether active credential for the model uses OAuth. - `ctx.modelRegistry.getProvider(providerId)?.auth.oauth?.isSubscription === true` — Detects providers that declare flat-rate subscription-backed OAuth. - `ctx.scopedModels` — Active session model constraints. - **Billing Classification Resolution Logic**: 1. If `userBillingPolicy.overrides[modelId]` or `userBillingPolicy.overrides[providerId]` is defined, use the user's explicit policy class. 2. If `ctx.modelRegistry.isUsingOAuth(model)` is `true` AND `provider.auth.oauth?.isSubscription === true`, classify as `"subscription"`. 3. For all API-key authenticated models, ambiguous OAuth providers, or unmapped accounts, **default to `"metered"`**. - **Shipped Conservative Default Policy (`config/billing-policy.json`)**: *There is intentionally no static provider billing-class map. Only live subscription OAuth metadata or an explicit user override can classify a model as `subscription`; every ambiguous provider defaults to `metered`.* ```json { "userOverrides": {}, "meteredSpendLimitUsdPerSession": 2.00, "requireConfirmationOnMeteredFallback": true } ``` ### B. Scored Routing Algorithm When evaluating eligible models for an agent persona: $$\text{Eligible} = \text{Available} \land \text{Authenticated} \land (\text{ContextWindow} \ge \text{ReqTokens}) \land \text{CapabilitiesSatisfied}$$ $$\text{Score} = \text{TaskFitScore} + \text{SubscriptionBonus}(+100) + \text{ThroughputScore} - \text{EstimatedCostPenalty} - \text{FailurePenalty}$$ ```typescript export interface ModelRoutingDecision { agentName: string; providerId: string; modelId: string; billingClass: "subscription" | "metered"; thinkingLevel?: "low" | "medium" | "high"; estimatedCostUsd: number; fallbackDecisions: Array<{ providerId: string; modelId: string; billingClass: "subscription" | "metered" }>; } ``` ### C. Execution Controls: Pre-Dispatch Budget Cap, Stall Detection & Pre-Mutation Failover 1. **Hard Pre-Dispatch Budget Gate**: If a selected model is `metered`, the Conductor estimates task cost. If `sessionAccumulatedSpend + estimatedCostUsd > meteredSpendLimitUsdPerSession` or if `requireConfirmationOnMeteredFallback` is active, task dispatch **hard-blocks** and prompts the user in the TUI for authorization before dispatching the subprocess. 2. **Streaming Stall Detection**: The dispatcher monitors the live stdout event stream. If **0 tokens/events are emitted for >30 seconds**, the subprocess is aborted via `tree-kill` and marked for failover. 3. **Pre-Mutation Failover Restriction**: Automatic failover to fallback models is **strictly restricted to the pre-mutation stage** (before any file edits or commands execute). Once tool execution begins, errors transition through the durable state machine to prevent duplicate side effects. 4. **Failure Classification**: - **Retryable** (HTTP 429, connection reset, 502/503/504, streaming stall): Increments failover counter; attempts fallback model. - **Terminal** (Auth invalid, context length exceeded, schema rejection, policy block): Halts task immediately with actionable error diagnostics. --- ## 4. Subprocess Execution, Sandboxing & Special Builder Safety ### A. Transport Strategy - **Phase 1 (MVP)**: Uses Pi's one-shot JSON event stream (`pi --mode json -p --no-session --model `). Captures streaming progress, tool events, and structured usage telemetry. - **Phase 2 (Persistent Workers)**: Introduces bidirectional RPC mode (`pi --mode rpc`) for long-running worker pools, interactive steering, and sub-session reuse. ### B. Dedicated Read-Only Search Tools (`sandboxed_fd` & `sandboxed_rg`) To ensure least privilege without exposing a shell to the Explorer: - **No-Shell Spawning**: Spawns binaries via `child_process.execFile` or `spawn` with explicit argument arrays (bypassing `/bin/sh`). - **Path Confinement**: Enforces search root strictly within the workspace `cwd` (rejects `..` or absolute paths outside repo). - **Prohibited Flags**: Disallows execution flags (`--exec`, `-x`, `--pre`, `--search-zip`). - **Execution Limits**: 10s execution timeout and 50KB stdout buffer cap. ### C. Special Builder Safety Controls The Special Builder is strictly governed to prevent accidental filesystem or environment damage: 1. **Interactive Human Gate**: Any destructive or system command requires explicit user confirmation in the TUI before execution. 2. **Protected Path Blocklist**: Shell commands and file operations are strictly prohibited from touching: - `~/.ssh/`, `~/.aws/`, `~/.gnupg/` - `.git/` (internal metadata) - `/etc/`, `/var/`, `/System/`, `/usr/` - Host environment startup files (`~/.zshrc`, `~/.bashrc`, `~/.zshenv`) 3. **Dry-Run Preview & Diff Review**: Before executing migration scripts or multi-file mutations, the agent must output a dry-run diff for inspection. 4. **Automated Rollback Scripts**: Special Builder generates an inverse rollback script (`.pi/runs//rollback-.sh`) prior to executing stateful operations. --- ## 5. Workspace Isolation, Worktrees & Full-Range Concurrency Safety ### A. Real Temporary Worktree Paths & Clean Workspace Enforcement Because sub-agents run in separate child processes, process-local mutexes (`withFileMutationQueue`) cannot coordinate independent child processes. Mutating builders are isolated in dedicated Git worktrees: ``` [Conductor Task Allocation] │ ├── Preflight: Verify Git repo & check clean working tree │ ├───────────────────────────────────────────────┐ ▼ ▼ [Builder 1: Worktree / Task A] [Builder 2: Worktree / Task B] Path Lease: ['src/components/**'] Path Lease: ['src/api/**'] Worktree: .pi/worktrees//task-A Worktree: .pi/worktrees//task-B Branch: sagent//task-A Branch: sagent//task-B │ │ └───────────────────────┬───────────────────────┘ ▼ [Quality Assurer Gate] • Runs linters & test suites on worktree branch • Max 2 rework cycles allowed │ (PASS) ▼ [Conductor Validation & Merge] • 1. Full-Range Lease Check: git diff --name-only .. • 2. Fast-Forward Task A into active branch • 3. Rebase Task B on updated HEAD • 4. Rerun QA Gate & Lease Check on rebased Task B • 5. Fast-Forward Task B into active branch • 6. Remove worktrees & prune branches ``` ### B. Concurrency, Dirty-Tree Refusal & Rebase Merge Rules 1. **Preflight & Non-Git Fallback**: - Conductor checks `git rev-parse --is-inside-work-tree`. - *Non-Git Workspaces*: Fallback to sequential execution with file-level OS locks; parallel mutating builders are disabled. 2. **Strict Clean-Workspace Policy for Mutating Tasks**: - Conductor checks `git status --porcelain`. - If the working tree is dirty (has staged, unstaged, or untracked changes), the Conductor **strictly refuses mutating worktree execution and merges**. - Output message: *"Working tree contains uncommitted changes. Please commit or stash changes before running mutating tasks."* - *Read-only agents* (`Explorer`, `Architect`, `Librarian` query tasks) are permitted on dirty workspaces. 3. **Deterministic Baseline Commit**: - Conductor records `baselineCommit = git rev-parse HEAD`. Every task branch `sagent//task-` is created from `baselineCommit`. 4. **Full-Range Path Lease Validation**: - Before dispatch, Conductor checks path lease globs. If two tasks have overlapping file paths, they are scheduled **sequentially**. - **Enforced Validation**: When a builder completes, the Conductor inspects the entire commit range: `git diff --name-only ..` If any modified file violates the assigned path lease, the merge is **REJECTED** and the task fails. 5. **Post-Rebase Lease & QA Revalidation**: - When Task A merges into `HEAD`, Task B must be rebased onto the newly advanced `HEAD` commit. - Because rebasing creates a newly untested artifact, the Conductor **must rerun lease validation and rerun the QA Gate test suite on the rebased Task B branch** before integrating into `HEAD`. 6. **Atomic Pre-Merge Base Check**: - Conductor verifies active `HEAD` matches the expected merge base before fast-forward or squash merge. 7. **Crash Recovery & Worktree Cleanup**: - On session startup or extension load, `worktree-manager.ts` scans `.pi/worktrees/` and runs `git worktree prune` to clean up any orphaned worktrees from previous crashes. --- ## 6. Durable DAG Orchestrator & Task Lifecycle ### A. Explicit 7-State Task State Machine ``` ┌──────────────┐ │ PLANNED │ └──────┬───────┘ │ (Dependencies Met & Lease Acquired) ▼ ┌──────────────┐ │ READY │ └──────┬───────┘ │ (Subprocess Spawned) ▼ ┌──────────────┐ ┌─────────►│ RUNNING ├────────────────┐ │ └──────┬───────┘ │ │ │ │ (Rework ≤ 2) (QA Pass & Merged) (Terminal Error / Rework > 2) │ │ │ │ ▼ ▼ ┌─────┴─────┐ ┌──────────────┐ ┌──────────────┐ │ RETRYING │ │ SUCCEEDED │ │ FAILED │ └───────────┘ └──────────────┘ └──────────────┘ │ │ (Cancel Event) (Needs Guidance) │ │ ▼ ▼ ┌──────────────┐ ┌──────────────┐ │ CANCELED │ │ BLOCKED │ └──────────────┘ └──────────────┘ ``` ### B. Scheduling Classes & Tracked Background Nodes - **Critical Path Nodes** (`Explorer`, `Architect`, `Builder`, `QA`): High priority, blocking execution. - **Tracked Background Nodes** (`Librarian`, `Accountant`, `Principal`): **Low-priority scheduling class** using the *exact same 7-state lifecycle*. They run with heartbeats, 60s lease timeouts, visible status in TUI HUD, and graceful teardown on session exit. ### C. Complete Manifest Schema & Atomic fsync Persistence - **Single-Writer Concurrency**: Conductor serializes all manifest updates through an in-process mutex queue (`manifestMutex`). - **Atomic fsync Procedure**: 1. Write serialized manifest to `.pi/runs//manifest.json.tmp`. 2. Call `fs.fsync()` on the file handle to flush file data to disk. 3. Atomically rename `manifest.json.tmp` -> `manifest.json`. 4. Open directory handle `.pi/runs//` and call `fs.fsync()` on directory handle to flush directory metadata. ```typescript export interface RunManifest { schemaVersion: "1.0.0"; manifestRevision: number; runId: string; sessionFile: string; createdAt: number; updatedAt: number; baselineCommit: string; targetBranch: string; cancellation: { requested: boolean; reason?: string; requestedAt?: number; }; recovery: { ownerPid: number; attemptCount: number; lastRecoveryAt?: number; }; tasks: Record; } ``` --- ## 7. Knowledge Hierarchy, Credential Safety & Guaranteed Local Vector Store ### A. Extension-Owned Profile Resolution & Path Traversal Safety Pi does not have native `--profile` or `defaultProfile` flags. The extension owns profile resolution: - **Authoritative Resolution Order**: 1. Extension command flag: `/sagent run --profile ` 2. Environment variable: `PI_PROFILE=` 3. Extension config: `~/.pi/agent/extensions/sagent/config/settings.json` -> `"defaultProfile"` 4. Fallback: `"default"` - **Path Traversal Validation**: Validates `profile` strictly matches `/^[a-zA-Z0-9_-]+$/` and resolves strictly within `~/.pi/profiles/`, rejecting `..` or external symlinks. - **Clean Environment Variable Propagation**: - The extension **never mutates `PI_CODING_AGENT_DIR`** unless set by the parent environment. - It exports `PI_PROFILE=` to child subprocesses. - **Three Distinct Directory Roots**: - *Pi Runtime Agent Directory*: `$PI_CODING_AGENT_DIR` (or `~/.pi/agent/`) for Pi core runtime. - *Fixed Global Knowledge Root*: Always `~/.pi/agent/` (accessible across all profiles). - *Active Profile Knowledge Root*: `~/.pi/profiles/${PI_PROFILE}/`. --- ### B. Canonical Storage & Guaranteed Local Vector Cache Root ``` ┌────────────────────────────────────────────────────────────────────────┐ │ Canonical Source of Truth (Human-Readable & Cloud-Synced) │ │ • Project: `${cwd}/.pi/AGENTS.md` + `${cwd}/.pi/knowledge-events.jsonl`│ │ • Profile: `~/.pi/profiles/${p}/AGENTS.md` + `knowledge-events.jsonl` │ │ • Global: `~/.pi/agent/AGENTS.md` + `~/.pi/agent/knowledge-events.jsonl`│ └───────────────────────────────────┬────────────────────────────────────┘ │ (Deterministic local rebuild on startup/delta) ▼ ┌────────────────────────────────────────────────────────────────────────┐ │ Guaranteed Local Derived Vector DBs (Strictly Local System Cache Root) │ │ Root: `~/.cache/sagent/vectors/` (Outside all Cloud-Sync Folders) │ │ • Project Cache: `~/.cache/sagent/vectors/projects//db.sqlite` │ │ • Profile Cache: `~/.cache/sagent/vectors/profiles/${p}/db.sqlite` │ │ • Global Cache: `~/.cache/sagent/vectors/global/db.sqlite` │ └────────────────────────────────────────────────────────────────────────┘ ``` --- ### C. Knowledge Lifecycle: Stable Chunk IDs, Tombstones & Provenance 1. **Pre-Write Secret Scanning (Defense-in-Depth)**: - Regex scanner + high-entropy scanner inspects text before writing to `AGENTS.md` or JSONL. - Quarantines API keys, AWS tokens, private keys, and JWTs, replacing them with symbolic references. 2. **Stable Chunk IDs & Deduplication**: - Chunks assigned deterministic UUIDv5 IDs: `uuid5(scope, source_file + heading + sha256(content))`. 3. **Idempotent Transactional Migration**: - When `AGENTS.md` exceeds **300 lines or 15k tokens**, Principal triggers Librarian: 1. Append older sections to `knowledge-events.jsonl` with UUIDs and timestamp. 2. Atomically rewrite `AGENTS.md` keeping only hot rules and active summaries. 3. Insert embeddings into local `db.sqlite`. 4. Recovers deterministically from `knowledge-events.jsonl` on startup if interrupted. 4. **Deletions & Tombstones**: - Removed rules record `{"type": "tombstone", "id": "", ...}` in JSONL, prompting vector store deletion. 5. **Re-Embedding & Versioning**: - Vector table records `embedding_model` (e.g. `bge-small-en-v1.5:v1`) and schema version; drops and rebuilds local index automatically on model change. 6. **Pre-Dispatch Retrieval Pipeline**: - Conductor queries vector store across scopes before dispatch: $$\text{Results} = \text{VectorSearch}(\text{query}, \text{scopes}=[\text{Project}, \text{Profile}, \text{Global}], \text{topK}=5)$$ - Injected into child prompts with clear provenance tags: `[KNOWLEDGE: PROJECT (confidence 0.88)] `. - **Precedence Law**: Vector results strictly **supplement** and never silently override explicit rules in active `AGENTS.md` files. --- ## 8. Governed Autonomous Self-Improvement Subsystem ### A. Threat Model & Trust Boundary (Candidate-Isolated Supervisor) `sagent` operates on developer workstations within the user's local security context: 1. **Trusted Domain**: The active, human-approved `sagent` extension and user-initiated interactive Pi sessions run in the user's desktop domain. 2. **Untrusted / Candidate Domain**: Unapproved candidate builders, generated code patches, and package install scripts are treated as **untrusted**. 3. **Supervisor Isolation**: The **Upgrade Supervisor** provides strict isolation against **unapproved candidate builders, generated code, and sandboxed test environments**. It is tamper-evident and isolated from candidate workspaces: ``` ~/.local/share/sagent/ ├── supervisor/ # Candidate-isolated supervisor (read-only in sandbox) │ ├── verify-digest.ts │ ├── run-sandboxed-tests.ts │ ├── baseline-tests/ # Candidate-immutable baseline acceptance tests │ ├── state-migrator.ts # Forward & backward state schema migrator │ ├── migration-journal.jsonl # Durable supervisor migration journal │ ├── audit-log.jsonl # Append-only audit log with SHA-256 HMAC chaining │ └── activate.ts ├── ipc/ # Authenticated Unix Domain Socket (chmod 0600) ├── repo/ # Trusted Git source repository for sagent ├── worktrees/candidate-/ # Isolated candidate build & test worktree ├── artifacts// # Built immutable extension releases └── state-backups// # Pre-upgrade manifest, config, and DB snapshots ~/.pi/agent/extensions/ └── sagent -> ~/.local/share/sagent/artifacts/ # Atomic Symlink ``` *Self-generated patches may propose supervisor changes only as an exported patch (`.patch`) requiring manual user inspection and installation outside `/sagent upgrade`.* --- ### B. Real Enforceable OS Sandboxing (Fail-Closed) Candidate dependency installation, compilation, and test execution run in an enforceable OS isolation backend: 1. **Backend Engine**: macOS Sandbox profile (`sandbox-exec` with strict profile), Linux namespace/seccomp (`bubblewrap`), or container/VM. 2. **Fail-Closed Policy**: If no supported OS sandbox backend is detected on the host system, candidate code builds and self-upgrades are **strictly refused**. 3. **Filesystem Isolation**: - Disposable root with read-only fixture mounts. - Exactly one writable scratch directory (`/tmp/scratch/`). - Host `~`, `~/.pi`, `~/.ssh`, `~/.aws`, sessions, sockets, and live profile directories are **strictly unmounted and inaccessible**. 4. **Zero Network by Default**: Network namespace is unshared/disabled during build, install, and test runs. 5. **Resource Limits**: Hard limits: 2 CPU cores, 1GB RAM, 50MB max output size, 60s wall-clock timeout. --- ### C. Dependency & Build Supply-Chain Controls All candidate code upgrades adhere to strict supply-chain policy: 1. **Frozen Lockfile Enforcement**: Builds execute using `bun install --frozen-lockfile` or `npm ci --ignore-scripts`. 2. **Lifecycle Scripts Prohibited**: Package install scripts (`preinstall`, `postinstall`) are strictly disabled by default. 3. **Registry & Dependency Policy**: Only packages from approved registries (e.g. `registry.npmjs.org`) are permitted. Local filesystem paths, raw Git URLs, or unpinned ranges are rejected. 4. **Vulnerability & License Audit**: Runs automated security scanning (`audit`) and license compliance checks. 5. **Deterministic Digest Calculation**: $$\text{CandidateDigest} = \text{SHA256}(\text{CommitHash} + \text{LockfileHash} + \text{ArtifactArchiveHash} + \text{TestReport} + \text{MigrationPlan})$$ --- ### D. Supervisor-Owned Human Approval & One-Time Challenge Token Protocol To prevent candidate or self-modified code from self-authorizing its own upgrades: 1. **Challenge Generation**: Upon successful sandboxed testing, the supervisor generates a cryptographically random One-Time Challenge Token bound strictly to the candidate digest: `ChallengeToken = CSPRNG_Hex(16)` 2. **Independent Approval Interface**: The supervisor displays the diff summary, migration plan, candidate digest, and token in the terminal. 3. **Atomic Consumption**: The user must explicitly execute: `/sagent upgrade ` (or via standalone TTY prompt). The supervisor atomically consumes ``, resolves it strictly to its pre-bound digest, and rejects any digest substitution. 4. **Append-Only Audit**: The supervisor records `(approver, timestamp, candidateDigest, challengeToken)` in the HMAC-chained audit log before activation. --- ### E. Multi-Point Health Verification, Migration Journaling & Out-of-Band Rollback 1. **Durable Migration Journal**: The supervisor maintains `migration-journal.jsonl` recording sequential phase checkpoints: `SNAPSHOT_CREATED` → `MIGRATE_UP_DONE` → `SYMLINK_SWAPPED` → `HEALTH_ACKNOWLEDGED` → `COMMITTED` 2. **Comprehensive 5-Point Health Criteria**: Upon reload, the new version must successfully pass all 5 verification points within **10 seconds**: 1. Extension entrypoint imported cleanly without unhandled exceptions. 2. All required tools and commands registered in Pi runtime (`/sagent`, `orchestrate`, `sandboxed_fd`, `sandboxed_rg`). 3. Configuration file opened and schema validated. 4. Manifest store and run history opened and schema validated. 5. Cryptographic IPC handshake with the supervisor acknowledged. 3. **Automated Crash Recovery**: If a crash occurs at any point during migration or if health acknowledgement fails within 10s: - Supervisor detects incomplete journal checkpoint. - Flips symlink back to ``. - Executes `migrateDown()` or restores pre-upgrade state snapshots (`state-backups//`). - Marks journal `ROLLED_BACK`. 4. **Out-of-Band Rescue Command**: `sagent-rescue rollback` provides a standalone terminal CLI utility capable of restoring the previous version if Pi cannot start. --- ### F. State-Schema Migration & Rollback Compatibility 1. **Versioned Migration Plans**: Every extension upgrade with schema changes includes forward (`migrateUp()`) and backward (`migrateDown()`) scripts for manifests, settings, agent frontmatter, and vector DBs. 2. **Sandbox Dry-Run**: The supervisor runs `migrateUp()` followed by `migrateDown()` against a temporary copy of active state inside the sandbox before presenting the upgrade to the user. 3. **N-1 Backwards Read Compatibility**: Code is designed to read N-1 schema versions gracefully during upgrade windows. 4. **Irreversible Migrations**: Any upgrade with irreversible data migrations requires a separate, explicit user confirmation warning before proceeding. --- ### G. Immutable Baseline Acceptance Test Suite Stored in `~/.local/share/sagent/supervisor/baseline-tests/`, this suite cannot be modified by candidate builders. Upgrades must pass 100% of baseline tests: 1. Candidate agent exclusion (`*.candidate.md` ignored by registry) 2. Tool privilege ceilings (no `bash` on synthesized agents) 3. Protected path blocklist & Special Builder approval gates 4. Model billing classification and hard spend cap enforcement 5. Worktree path lease enforcement and clean tree checks 6. Manifest single-writer crash recovery & directory fsync 7. Profile and knowledge isolation (no cross-profile leakage) 8. Supervisor/policy immutability & TOCTOU digest rejection 9. Sandbox escape denial 10. Multi-point health acknowledgement & automatic rollback 11. Migration forward and backward round-trip validation 12. Process-kill crash recovery across each migration checkpoint --- ### H. Churn Limits, Budgets & Documentation Governance 1. **Supervisor-Enforced Global Kill Switch**: - `enableAutonomousUpgrades: false` by default in supervisor configuration. - Can be toggled out-of-band via `sagent-supervisor --disable-upgrades`. Mutable extension code cannot override this setting. 2. **Daily Self-Improvement Budgets**: - Hard daily cap of **500k tokens / $1.00 USD** for candidate generation, sandboxed builds, and test runs. 3. **Proposal Deduplication & Cooldown**: - Minimum **24-hour cooldown** between upgrade proposals. - Proposals require SHA-256 fingerprinting of recurring task friction (minimum threshold: ≥3 distinct occurrences). - Maximum **1 active candidate** at any time. 4. **Documentation Scoping & Secret Scanning**: - Extension self-documentation (retrospectives, capability cards, changelogs) is written strictly inside candidate worktrees. - Pre-write secret scanning screens all documentation before commit. - Project repository documentation changes require standard worktree path leases and user review. 5. **Strict Persona vs Code Boundary**: - *Persona/Prompt changes*: Handled via `.candidate.md` -> `/sagent approve `. - *Tool schemas, executable code, or dependencies*: Must use the full **Supervised Code Upgrade Path** -> `/sagent upgrade `. --- ## 9. Extension Directory Layout ``` ~/.pi/agent/extensions/sagent/ ├── index.ts # Extension entrypoint, tool registration, CLI commands ├── package.json ├── config/ │ ├── billing-policy.json # Shipped conservative billing classes & caps │ └── settings.json # Concurrency limits, KB thresholds, paths, defaultProfile ├── core/ │ ├── conductor.ts # Durable DAG orchestrator & task state machine │ ├── router.ts # Scored Model & Auth Routing Engine with spend caps │ ├── dispatcher.ts # Subprocess spawner (JSON mode & RPC mode) │ ├── worktree-manager.ts # Git worktree lifecycle & path lease manager │ ├── secret-scanner.ts # Pre-write credential & entropy detector │ ├── supervisor-client.ts # Client interface to immutable upgrade supervisor │ └── pool.ts # Concurrency pool limiter (max 4 parallel builders) ├── tools/ │ ├── sandboxed-fd.ts # Sandboxed no-shell fd tool │ └── sandboxed-rg.ts # Sandboxed no-shell rg tool ├── knowledge/ │ ├── store.ts # Multi-tier scope resolver (Project -> Profile -> Global) │ ├── vector-db.ts # Local system cache SQLite + sqlite-vec engine │ ├── embeddings.ts # Local ONNX (bge-small) / API embedding client │ └── threshold-scaler.ts # Principal-driven AGENTS.md vectorizer & JSONL logger ├── agents/ │ ├── registry.ts # Agent discovery, candidate filtering, promotion │ └── builtins/ # Default Agent Personas (.md) │ ├── explorer.md │ ├── architect.md │ ├── conductor.md │ ├── builder.md │ ├── special-builder.md │ ├── qa.md │ ├── accountant.md │ ├── librarian.md │ └── principal.md └── tui/ ├── swarm-widget.ts # Multi-agent live streaming container ├── cost-tracker.ts # Token, spend cap, and cost HUD └── model-picker.ts # Interactive /sagent-models modal ``` --- ## 10. Implementation Roadmap - **Phase 1: Foundation & Sandboxing**: - Subagent extension using `pi --mode json` subprocess execution. - Implement sandboxed `sandboxed_fd` and `sandboxed_rg` tools. - Implement `secret-scanner.ts` with regex and entropy checks. - **Phase 2: Worktree Isolation & Router**: - Build `worktree-manager.ts` with baseline commit tracking, full-range lease diff validation (`..`), clean-tree enforcement, and post-rebase QA rerun. - Build `router.ts` consuming `ctx.modelRegistry`, `isUsingOAuth(model)`, and `provider.auth.oauth?.isSubscription === true` with hard spend caps. - **Phase 3: Durable DAG Orchestrator**: - Implement 7-state task machine with atomic journal persistence (`fsync` on temp file + directory). - Integrate QA Gate with bounded rework loops (max 2 cycles). - Add tracked low-priority background nodes with heartbeat and graceful teardown. - **Phase 4: Multi-Tier Knowledge & Local Cache Vector Index**: - Build 3-tier resolver with profile validation (`$PI_PROFILE`), clean env propagation, and cross-profile query isolation. - Implement `vector-db.ts` under `~/.cache/sagent/vectors/` with deterministic rebuild from `AGENTS.md` + `knowledge-events.jsonl`, chunk UUIDs, tombstones, and re-embedding versioning. - **Phase 5: Self-Improvement & Upgrades Subsystem**: - Implement self-documentation and candidate agent synthesis (`.candidate.md`) with human approval (`/sagent approve`). - Implement Candidate-Isolated Upgrade Supervisor in `~/.local/share/sagent/supervisor/`, fail-closed OS sandbox runner, digest/challenge token verification, migration journal checkpointing, multi-point health check, and out-of-band rescue CLI (`sagent-rescue rollback`). - Build live streaming TUI widget, cost HUD, and runtime model switcher modal.