TachiBot Infra Plan

tachibot-mcp + tachi-ui · researched 2026-07-03 · 51 Sonnet agents, 5 angles, 81-claim pool, 15 load-bearing claims adversarially verified (10 survived, 5 killed & corrected) · gitignored — internal only

1 · Where infra lives: in-repo, no IaC

verified Keeping infra config next to application code enables atomic commits across app+infra changes and is the practitioner consensus for small teams; separate infra repos are justified by privilege/audit boundaries that don't exist for a solo developer (Harness, HungryDevOps). PaaS config formats agree: verified fly.toml is designed to sit per-app in the app's own repo (Fly docs); Railway's railway.json/toml works the same way.

Concretely:

• tachi-ui/ keeps its Dockerfile + a fly.toml (or railway.toml) + optional compose.yaml for local runs.
• tachibot-mcp/ needs no deploy config today — it ships via npm/.mcpb and runs on user machines. When the hosted judge API arrives, it gets its own fly.toml in whichever repo hosts that service.
• No Terraform/OpenTofu/Pulumi/SST now. The 2026 small-team pattern is Docker (Compose) + a PaaS config file, or Kamal 2 for raw-VPS deploys. The claim "Terraform is explicitly called overkill for this" was killed in verification (source misquoted — the article actually calls Kubernetes overkill), but the practical takeaway stands on the other sources: at 1–2 small apps, a PaaS file is all the IaC you need. If resource count ever grows, pick OpenTofu (MPL license) over Terraform (BSL).

2 · Hosting tachi-ui

Constraint that drives everything: better-sqlite3 writes a file at process.cwd() — the platform must offer a persistent disk (or you rearchitect the data layer).

Data layer: single-node + backup, nothing fancier

• verified Turso free tier: 500M row reads/mo, 10M writes/mo, 5GB across 100 DBs; first paid tier $4.99/mo; libSQL embedded replicas give local-disk-speed reads with a durable cloud primary (pricing). The managed escape hatch if self-managed backups ever chafe.
• LiteFS: the researched claim that it's "dead, avoid" was killed — LiteFS Cloud (the managed backup add-on) was sunset Oct 2024, but LiteFS itself is maintained; Fly's own guidance is topology-based: Litestream for single-node, LiteFS for multi-node. tachi-ui is single-node → Litestream to Tigris/R2/S3.

Pre-deploy hardening (found in local recon, do before any deploy)

• Docker CMD currently runs vite preview for the SPA — replace with Hono serving the built dist/ statically on one port (drops the second exposed port and a dev-server in production).
• DB path is hardcoded to process.cwd() (src/db/connection.ts:101) — make it env-configurable (DATABASE_PATH=/data/tachi.db) so it lands on the mounted volume.
• Ports are inconsistent (Dockerfile EXPOSEs 3005, package.json docker:run maps 3002) — pick one.

3 · Where models come from

3a · tachibot-mcp today (local-first, users' own keys)

Keep the current hybrid — it matches what the research says is optimal.

• Direct APIs for the big four (Perplexity, Grok, OpenAI, Gemini): provider-side prompt caching is the decisive reason — discounts OpenRouter's abstraction doesn't cleanly expose. Verified cache pricing: Grok 4.3 cached input $0.20/1M (~85% off $1.25); GPT-5.5 cached input $0.50/1M (90% off $5.00); Gemini 3.1 Pro cached input $0.20/1M (90% off $2.00). Batch APIs add a stacking 50% where supported.
• OpenRouter for the long tail (Kimi, DeepSeek, GLM, MiniMax, StepFun, Qwen, ERNIE): verified no per-token markup on most models; the real costs are a 5.5% credit-purchase fee (5% crypto) and a 5% BYOK fee waived up to a free monthly quota — OpenRouter's own docs state that quota inconsistently ($25k–$200k/mo list-price inference on the pricing page vs. 1M free BYOK requests/mo in the BYOK docs), so re-check at time of use. One aggregator key for seven providers is worth 5.5% to end users.
• The popular "$500/mo → OpenRouter, $2,000/mo → direct" rule was killed for this project: it's calibrated for a company aggregating spend on one key. TachiBot users each bring their own keys and will never individually hit direct-API tier thresholds — the hybrid is chosen for model access and caching, not spend tiers.
• Ship no gateway. A LiteLLM/Portkey proxy is a second service end users would have to run alongside a stdio MCP server — wrong shape for an npm-distributed local tool. Users who want routing control can point OpenRouter BYOK at their own provider keys.
• Reliability note verified: OpenRouter had two acknowledged outages Feb 17 & 19 2026 (same root cause, 80–90% request failures at peak). TachiBot's architecture already absorbs this — the big four are direct, so an OpenRouter outage degrades only the long-tail tools. Keep it that way; don't consolidate the big four onto OpenRouter.

3b · Future hosted judge API (developer-paid keys)

• Start with the same hybrid on your own keys: direct big four + OpenRouter long tail. Add hard spend caps in app logic from day one.
• Defer a gateway. verified (2/3) LiteLLM is free to self-host but carries $100–400/mo hidden costs (managed Postgres with backups/HA) plus ongoing DevOps — only worth it once you have multi-tenant per-key budgets to enforce. Its virtual-keys/budgets system is the most battle-tested when that day comes; Portkey OSS is the runner-up.
• Managed zero-markup gateways (Vercel AI Gateway, Cloudflare AI Gateway) are only compelling if the service already lives on that platform; Cloudflare's uniquely offers dollar-denominated spend limits and edge caching. search-stage, not adversarially verified
• Avoid Helicone for new dependencies: acquired by Mintlify (announced Mar 2026) and reportedly in maintenance mode. search-stage, not adversarially verified
• Claude via OpenRouter reportedly carries a ~100% markup vs. direct — moot today (TachiBot doesn't call Claude; Claude is the host), but if the judge API ever adds Claude jurors, go direct to Anthropic. search-stage, not adversarially verified
• xAI reportedly offers up to $175/mo in free API credits via data-sharing opt-in — useful for dev/test spend if acceptable. search-stage, not adversarially verified

Verified price sheet (as of 2026-07-03)

4 · Phased roadmap

5 · Selling it: hosted tachi-ui, credits + BYOK (decided 2026-07-03)

The decision: host and sell tachi-ui as a service. The thinner judge-API wedge (prior council's pick) stays on the shelf as Phase 2 of the roadmap — hosted tachi-ui is the product now, and it can later call the judge API as a pipeline node.

Backend: extend, don't build

tachi-ui already has a backend — the Hono server in src/server/ (16 routes: pipeline execution, AI providers, SSE, tools, MCP). It has zero auth, users/orgs, or billing. The commercial layer goes into the same repo, same server, one deployable — no separate backend repo/service at this scale (consistent with §1's in-repo verdict):

• Auth + organizations (better-auth, or Clerk free to 10k MAU) — the biggest new code surface
• Tenancy: one SQLite file at cwd today → add org_id to the Drizzle schema, or per-tenant Turso DBs (free tier covers 100 databases — the native fit)
• Encrypted per-tenant storage of users' provider keys (the BYOK trust surface)
• Stripe Checkout (one-time credit packs — far less code than metered subscriptions) + per-tenant rate limits
• The §2 Phase-0 hardening in the same pass: Hono serves dist/, env-var DATABASE_PATH, one port

Tiers — keep BYOK permanently

BYOK and credits are two tiers of one product, not competing options. BYOK costs nothing to keep (it's the current architecture), is the acquisition funnel (fed by tachibot-mcp's ~1,600/mo npm downloads), and insulates the business — a provider limiting your account doesn't touch BYOK users. Removing it would make you sole payment processor for all inference: maximum float + abuse risk.

Mental model: seats are the profit; credits are the onboarding ramp. Credits alone rarely make a solo product profitable early — but they remove the single biggest signup barrier ("go get API keys"), and those users later buy seats.

Credits mechanics (the safe version)

• Prepaid only, never postpaid. Users buy packs, you deduct per request, hard-stop at zero. Their money lands before your provider spend — float shrinks to a $200–500 provider prepay buffer covering the few-day Stripe payout lag.
• Metering is app code, not infrastructure. The Vercel AI SDK returns token usage per call; keep a per-model price table (provider list + markup), write a ledger row, deduct. No LiteLLM/gateway until multi-tenant budgets are real (§3b).
• Margin: 25–40% over provider list is the normal band. Real extra margin is caching — pipelines share big prompt prefixes; you charge full input price, you pay the cached price (90% off on GPT-5.5/Gemini, ~85% on Grok — §3 price sheet). A $25 pack ≈ $6–10 gross after provider costs and Stripe's ~2.9% + 30¢.
• Abuse controls from day one: card required before any credits (or max ~$1 free grant), per-minute spend cap, model allowlist per tier.

Credit-ledger schema sketch (Drizzle)

orgs            id · name · plan (free|pro) · created_at
users           id · org_id · email · role
provider_keys   id · org_id · provider · key_encrypted · created_at        // BYOK, encrypt at rest
credit_ledger   id · org_id · delta_usd_micros · reason (purchase|usage|grant|refund)
                · request_id · model · tokens_in · tokens_out · created_at
                // balance = SUM(delta) per org — append-only, never UPDATE
model_prices    model · in_usd_per_mtok · out_usd_per_mtok · markup_pct · updated_at

Append-only ledger: balance is a SUM, disputes are auditable, refunds are negative rows. Store micros (integers), never floats.

July → August checklist (salary lands ~August)

Startup cost summary

6 · Provenance & limitations

Method: 5 Sonnet search agents (one per angle: hosting, infra-config placement, OpenRouter, gateways, direct-API economics + SQLite-on-PaaS) → 81-claim pool → Sonnet triage picked 15 load-bearing claims → each claim attacked by 3 adversarial fact-checkers (currency / primary-source accuracy / context lenses) that fetched primary sources; ≥2 refutes kills. 10 survived, 5 killed. 51 agents, ~1.79M tokens, 311 tool calls, ~8 min.

Limitations: the dedicated deep-read stage failed on a script bug (sandbox lacks the URL constructor; used only for display labels), so the claim pool came from search agents alone — partially compensated by verifiers fetching primary sources directly. Items tagged not adversarially verified come from search-stage summaries (notably: Helicone/Mintlify maintenance mode, Claude's ~100% OpenRouter markup, xAI free credits, Portkey/Vercel/Cloudflare gateway details) — re-verify before acting on those specifically.