# Rollbridge TODO

This roadmap tracks planned Rollbridge features and documentation. Rollbridge should stay deploy-tool agnostic: Capistrano, shell scripts, CI, or other deploy systems should call Rollbridge through CLI commands instead of Rollbridge shipping Capistrano tasks or plugins.

## Current Baseline

- [x] Stable local HTTP/WebSocket proxy in front of one active release.
- [x] Unix socket control API used by the CLI.
- [x] `daemon`, `ensure-daemon`, `deploy`, `status`, `stop`, and `shutdown` commands.
- [x] Per-release process startup with templated `releasePath`, `releaseId`, `revision`, and ports.
- [x] Process policies for `proxied`, `companion`, `singleton`, and `service`.
- [x] HTTP health check before switching traffic to a new web process.
- [x] Drain old HTTP/WebSocket connections before stopping previous release processes.
- [x] Restart crashed active release processes after `restartDelayMs`.
- [x] Restart crashed daemon-wide service processes from the latest successful release template.
- [x] Graceful `SIGTERM` followed by `SIGKILL` after timeout.
- [x] TensorBuzz production example config.
- [x] ESLint, TypeScript `checkJs`, TensorBuzz CI, and `release:patch`.

## Major Features

- [x] Memory supervision.
  - [x] Add per-process memory config with an RSS limit, check interval, warning threshold, and restart policy.
  - [x] Measure the managed process tree, not only the shell wrapper PID. (Sums RSS across the process group via `/proc`.)
  - [x] Report memory stats and last memory-triggered restart in `status`.
  - [x] Restart memory-heavy workers gracefully when possible, with a forced stop timeout.
  - [x] Add tests with a fixture process that allocates memory above the configured limit.
- [x] Worker auto-restart and restart policy controls.
  - [x] Add config for max restarts, restart window, exponential backoff, and disabled restart behavior (per-process `restart` policy).
  - [x] Distinguish crash restarts, deploy replacements, manual restarts, and memory restarts in status/events. (Per-process `lastStartReason` + a `reason` on the `process started` event; the `memory` reason is wired and fires once memory supervision restarts a process.)
  - [x] Add a `restart` CLI command for a single process, a policy group, or all non-proxied workers.
  - [x] Keep restart behavior safe for job workers by using lifecycle hooks before termination. (Manual restart, memory restart, and deploy-drain stops all run the `lifecycle` hooks via `stop()`.)
- [x] Graceful process-stop controls for job workers.
  - [x] Add generic lifecycle hooks such as `quietCommand`, `drainCommand`, `drainTimeoutMs`, and `stopCommand` (per-process `lifecycle`).
  - [x] Support signal-only lifecycle steps for workers that can quiet on a Unix signal. (Per-process `stopSignal`; sent before the `SIGKILL`-after-`gracefulStopMs` fallback.)
  - [x] Add a non-blocking drain mode so a worker can quiesce at release retirement independently of the HTTP/WebSocket connection drain (`nonBlockingDrain`). This control alone does not provide durable retired-generation supervision.
  - [x] Document the required Velocious release-generation contract (`docs/velocious.md` and `docs/workers.md`) without treating documentation as proof that the runtime implements it.
  - [x] Implement same-owner jobs-main retirement, concurrent generations, independent drains, and live release-reference reporting.
  - [x] Implement opt-in durable guardian recovery for exact same-authority daemon process replacement without stopping retained generations.
  - [x] Implement guardian-fenced atomic incompatible owner/config/control-socket/package/runtime replacement without stopping retained generations.
    - [x] Add the authenticated one-time disruptive bridge for pre-replacement guardians; all protocol-capable replacements remain atomic.
- [x] Replicas and stable worker indexes. (Supported on port-less `companion` processes; `proxied`/`singleton`/ported processes stay single.)
  - [x] Allow one process config to start multiple replicas (`replicas`, companion-only for now).
  - [x] Expose `ROLLBRIDGE_REPLICA_INDEX`, replica count, and per-replica template context (`{{replicaIndex}}`/`{{replicaCount}}`).
  - [x] Restart or stop one replica without affecting the rest (`rollbridge restart --process worker#0`).
  - [x] Preserve readable status output for replica groups (each instance shown as `<id>#<index>`).
- [x] Persistent daemon state and recovery.
  - [x] Persist active release, draining releases, process metadata, counters, and recent events (opt-in `statePath`; atomic snapshot on change + periodic).
  - [x] Reconnect to guardian-owned child processes when `ownerRecovery` is enabled; otherwise `status.orphans` retains advisory legacy behavior.
  - [x] Detect and report orphaned Rollbridge-managed processes. (On startup, reports persisted process pids that are still alive; advisory, see `statePath`.)
  - [x] Add a recovery mode for safe startup after daemon crash or machine reboot. (`rollbridge recover` lists orphaned processes from the persisted state and, with `--force`, stops them and clears the state; refuses while a daemon is running.)
- [x] Rollback support.
  - [x] Keep enough release metadata to switch traffic back to a previous healthy release.
  - [x] Add a `rollback` CLI command that health-checks the target before switching.
  - [x] Define how rollback interacts with singleton workers and draining releases. (Reuses the deploy flow: replaces singletons and drains the current release.)
  - [x] Document migration constraints for rollback.
- [x] Observability and diagnostics.
  - [x] Add structured event history for deploys, switches, stops, crashes, memory restarts, and failed commands. (In-memory `EventLog` tapping the daemon logger; memory-restart events populate once memory supervision logs them.)
  - [x] Add restart counters and uptime to status (exit reasons already reported via `exitCode`/`exitSignal`/`state`).
  - [x] Add memory stats and child-process-tree details to status (with memory supervision). (`rssBytes`/`memoryRestarts`/`lastMemoryRestartAt` plus `children`: the sampled process tree with each member's pid, command, and RSS.)
  - [x] Add a `logs` CLI command (recent per-process output from status).
  - [x] Add an `events` CLI command (after structured event history lands).
  - [x] Add optional file logging with rotation guidance (`docs/logging.md`; daemon log file via `--daemon-log-path`, logrotate `copytruncate`).
  - [x] Add machine-readable JSON output for all CLI commands (data commands print JSON; `validate`/`doctor`/`logs` take `--json`).
- [x] Config validation and doctoring.
  - [x] Add `validate` to parse config and report all config errors without starting the daemon.
  - [x] Add `doctor` to check config validity, control socket reachability, proxy port availability, and control-socket directory writability.
  - [x] Extend `doctor` with state-path checks: state-path directory writability and orphaned-process reporting from a prior state file.
  - [x] Extend `doctor` with process-command and release-path checks once those are resolvable (they need per-release rendered templates, which only exist at deploy time). (`rollbridge doctor --release-path <path>` renders each process's command/cwd/env against that release and checks the release directory, template resolvability, and rendered working directories; uses representative ports and replica index 0.)
  - [x] Validate duplicate process IDs, missing ports on proxied processes, invalid ranges, and the single-proxied-process policy rule.
  - [x] Validate unsupported lifecycle-hook combinations once worker lifecycle hooks land. (`lifecycle.drainCommand` requires a positive `drainTimeoutMs`; `nonBlockingDrain` is companion-only; a `lifecycle.stopCommand` may not be combined with a custom `stopSignal`, since the command runs instead of the signal.)
  - [x] Include example fixes in validation output.

## Minor Features

- [x] Add a control-socket permission option (`control.mode`) for shared deploy users.
- [x] Add control-socket owner/group options for shared deploy users (`control.owner`/`control.group`, numeric id or name resolved via `/etc/passwd`/`/etc/group`).
- [x] Make stale control socket diagnostics clearer when another daemon is still alive.
- [x] Add old-release cleanup policies by age, count, and stopped state (`releaseRetention`).
- [x] Add port allocation diagnostics when a range is exhausted.
- [x] Add an optional startup delay (`health.startDelayMs`) before health checks begin.
- [x] Add process output retention config instead of a fixed recent-log count.
- [x] Add environment variable interpolation from the daemon environment.
- [x] Add `--config` default lookup resolving to `rollbridge.js` when no path is given.
- [x] Add shell completion generation for common shells (`rollbridge completion bash|zsh`).
- [x] Add npm package metadata such as repository, license, bugs, and homepage.
- [x] Add systemd service examples for the Rollbridge daemon.
- [x] Add tests for malformed control socket JSON and unknown control commands.
- [x] Add tests for duplicate IDs and singleton replacement failure behavior.
- [x] Add tests for proxy behavior when the active release exits unexpectedly.

## Documentation TODO

- [x] Write a full config reference covering every field, default, and template variable (`docs/config.md`).
- [x] Write a CLI reference for `daemon`, `ensure-daemon`, `deploy`, `status`, `stop`, `shutdown`, and future commands (`docs/cli.md`).
- [x] Expand process policy docs with deployment examples for `proxied`, `companion`, `singleton`, and `service`.
- [x] Document memory checks and auto-restart behavior after the feature lands (`docs/config.md` → `processes[].memory`).
- [x] Document the required background-job generation pattern (`docs/workers.md`: release-scoped jobs-main + worker pool, independent quiescence, durable retained supervision, and deploy completion that does not wait for drains).
- [x] Document worker lifecycle hooks (`docs/config.md` → `processes[].lifecycle`, `docs/workers.md`).
- [x] Add a Velocious deployment guide with Beacon, background-jobs-main, background-jobs-worker, and web process examples (`docs/velocious.md`).
- [x] Add an Nginx guide with WebSocket headers, timeouts, and common failure modes (`docs/nginx.md`).
- [x] Add deploy-tool recipes that call Rollbridge CLI commands directly (`docs/deploy-recipes.md`).
- [x] Add a Capistrano recipe showing shell commands only; do not add a Capistrano plugin or Rollbridge-specific Capistrano tasks (`docs/deploy-recipes.md`).
- [x] Add a TensorBuzz-specific runbook for current production ports, external services, deploy ordering, and rollback constraints (`docs/tensorbuzz-runbook.md`).
- [x] Add troubleshooting docs for health-check failures, port conflicts, stale sockets, crash loops, and stuck draining releases (`docs/troubleshooting.md`).
- [x] Add a release checklist for maintainers using `npm run release:patch` (`docs/releasing.md`).