# Operations Runbook

This runbook explains how the tenant billing demo is operated and why the supporting prose matters for graph retrieval. In real repositories, a large share of useful project knowledge lives in documents like this one: onboarding notes, deployment steps, incident checklists, support guidance, and finance process explanations. Agents that work without structure often need to ingest these files together with the code because they cannot tell ahead of time which parts are relevant to the current question. That quickly grows token usage, especially when the repository contains both source files and dense operational text.

For onboarding, new engineers are asked to trace five recurring workflows. First, confirm how password login creates a tenant session. Second, identify the module responsible for sending invoice receipts. Third, find the code path that runs monthly billing close. Fourth, trace how the monthly revenue report is built. Fifth, find the source of truth for tenant context shared between auth and billing. Those are not arbitrary prompts. They are the same questions that come up in code reviews, support incidents, bug triage, and audits. The benchmark file mirrors this runbook so the proof measures something a team would actually care about.

During a month-end close, operators care about correctness, not just speed. The close job should use the current tenant context, produce a revenue report, and avoid accidentally sending duplicate notifications. The demo does not include a scheduler or a queue, but it preserves the shape of the workflow. A scheduled entrypoint calls the close job, the close job coordinates invoice and reporting logic, and the billing services reuse the same tenant-scoped structures that appear in auth. That shared shape gives the graph enough structure to answer operational questions with a focused set of nodes instead of requiring the agent to scan the whole repository.

Support investigations usually start from a symptom, not a module name. A customer says they logged in but did not see a session take effect. Another customer says they received an invoice but no receipt email. Finance asks where a monthly report total came from. In each case, the person answering the question needs both code and context. They need to know which modules participate, where responsibility is split, and what adjacent files might matter. The graph helps because it favors connected evidence over raw file count. Instead of reading every file that mentions "tenant" or "invoice", an agent can retrieve the specific services and bridge types that connect the workflow.

Operational documents also tend to include terms that overlap partially with code. Words like login, session, tenant, billing, monthly, report, receipt, and support appear in prose, source, and filenames. Without structure, that overlap can cause retrieval to either over-read the repo or mix explanatory text with implementation details in a noisy way. A graph does not remove the need for docs, but it gives the agent a better first cut: start from the entities and edges most likely to answer the question, then pull in the neighboring files and documents that explain those nodes. That is exactly the behavior the benchmark and eval commands are intended to make visible.

For demos and customer calls, this runbook gives you a practical script. Generate the graph for `examples/demo-repo`. Run `benchmark` against the checked-in question set to compare the full corpus with the graph-selected context bundle. Then run `eval` on the same question file to prove recall and ranking quality. The benchmark tells you whether the graph is cheaper to query than naively reading everything. The eval tells you whether the smaller bundle still contains the expected evidence. Users do not need to take marketing claims on faith because they can inspect the corpus, question set, and output in one repository.

The broader operational takeaway is that structure matters more as repositories accumulate prose, process, and side concerns. Code alone is rarely the full knowledge base. Teams also maintain runbooks, postmortems, architectural notes, and onboarding material. Those assets are valuable, but they make naive prompting more expensive. A graph-based workflow is most useful when it can keep those assets available without forcing every query to pay the full token cost of reading them all.