ax
Score any site's agent readiness from your terminal or CI — and watch real AI agents navigate it.
The open-source CLI for ora.
```
npx ax audit https://stripe.com
```
**No account, no API key.** Everything below works out of the box — the one exception is [`journey`](#for-ora-partners), which drives ora's partner platform.
Five commands:
- **`audit `** — run ora's hosted agent-readiness audit against a site: live progress, then a layered report of what passed, what's broken, and ora's ranked list of the highest-impact fixes (or `--json` for the raw contract payload). No account or API key needed. Gate CI with `--min-score`.
- **`webmcp-audit `** — audit a page's `document.modelContext` tools in a real browser. Works on any http(s) URL, localhost included, so you can check a surface before you publish it. Measures the page the way ora's capture worker does and has ora score it; nothing is stored, published, or ranked.
- **`deep-journey `** — run a real AI agent at a site on one of ora's curated tasks through the public journey API: no key, no workspace. A partner API key unlocks free-text tasks (`--task`) and a larger allowance.
- **`journey ""`** — send a real AI agent (claude-code, codex, …) at a site and watch its navigation live as a boxed node-graph, then get the scored insight, tokens, and cost. Partner-only: needs an `ORA_API_KEY` (see [For ora partners](#for-ora-partners)).
- **`skill [name]`** — list, print, or install ora's agent skills, digest-verified from the public registry.
## audit
```
ax audit [--min-score n] [--max-age s] [--force] [--tunnel ora] [--tunnel-cmd c] [--api-key k] [--json] [--show-passing] [--show-skipped]
```
```
stripe.com 78/100 B+
Stripe offers strong developer resource discoverability, but lacks an OpenAPI specification.
Discovery ██████░░░░ 6/9 passed · 1 skipped
✗ Issues (2)
┌───┬──────────────┬───────────────────────────┬──────────────────────────────┐
│ │ Check │ What's wrong │ How to fix │
├───┼──────────────┼───────────────────────────┼──────────────────────────────┤
│ ✗ │ sitemap.md │ status 404 │ Publish /sitemap.md listing… │
├───┼──────────────┼───────────────────────────┼──────────────────────────────┤
│ ⚠ │ robots.txt │ partially restricted │ Allow AI crawlers in robots… │
└───┴──────────────┴───────────────────────────┴──────────────────────────────┘
Top fixes (ranked by ora)
1. sitemap-md ≈ +4 pts
2. robots-txt ≈ +1.2 pts
cached result (43 min old) · pass --force for a fresh scan
Full report: https://ora.ai/stripe.com
```
| Flag | Effect |
|---|---|
| `--min-score ` | Exit `1` when the score is below `n` (0-100) — the CI gate |
| `--max-age ` | Accept a cached result up to `s` seconds old (server default 6h, clamped to 1-24h) |
| `--force` | Bypass the cache and rescan — spends the stricter 6/day force budget |
| `--tunnel ora` | Expose a local target through ora's own tunnel and audit it — needs `ORA_API_KEY` with the `tunnels:write` + `tunnels:connect` scopes (also read from `ORA_TUNNEL`) |
| `--tunnel-cmd ` | Bring your own tunnel: a command that exposes the local target and prints its public `https://` URL (also read from `ORA_TUNNEL_CMD`) |
| `--api-key ` | ora-issued scan API key — lifts every scan rate limit (also read from `ORA_SCAN_API_KEY`) |
| `--json` | The raw ora audit payload on stdout, exactly as the API served it |
| `--show-passing` | List every passing check, not just the per-layer summary bar |
| `--show-skipped` | Include not-applicable/pending checks with their reason |
The **Top fixes** list is ranked by ora server-side (non-bonus fixes first, then estimated score uplift) — the CLI renders it verbatim. All `≈ +N pts` figures are estimates. Check *tiers* (required / recommended / emerging) are advisory display metadata and never determine the score.
### CI gate
```yaml
- name: Agent-readiness gate
run: npx ax audit https://your-site.com --min-score 70
```
Exit codes are the contract:
| Code | Meaning |
|---|---|
| `0` | success (and score ≥ `--min-score` when given) |
| `1` | score below `--min-score` |
| `2` | usage error — bad flags, malformed URL, local target without a tunnel |
| `3` | API unreachable, timeout, or rate limit exhausted |
Budget notes: ora allows 30 scans + 6 `--force` scans per rolling 24h per IP (plus a 10/min burst limit). Results served from the freshness cache cost nothing, so a CI job that audits on every push stays well inside the budget — tune the window with `--max-age`, and reserve `--force` for verifying a fix you just deployed. High-volume callers can present an ora-issued scan API key (`--api-key` or `ORA_SCAN_API_KEY`), which exempts them from all of these limits; keys are issued manually by ora (no self-serve signup), and an unrecognized key silently falls back to the keyless limits rather than erroring. An auth-gated MCP target is not a failure: ora answers with an `MCP_AUTH_REQUIRED` error and no score; ax prints the notice (or the error object with `--json`), skips `--min-score`, and exits `0`.
### Auditing localhost (`--tunnel ora`, `--tunnel-cmd`)
A local dev server only exists on your machine, so ora can't reach it. The simplest option is to audit a publicly reachable deployment of the same code (e.g. a preview URL). To audit localhost itself, run it through a tunnel. The result is stored as **ephemeral** either way: excluded from ora's rankings and leaderboard, served with `Cache-Control: no-store`, never indexed, and deleted after 7 days.
**ora's own tunnel.** With an ora platform key (`ora_sk_…`, scopes `tunnels:write` + `tunnels:connect`) in `ORA_API_KEY`, `ax` creates a tunnel, connects it, audits the public hostname, and deletes the tunnel when done — nothing to install, nothing left behind:
```
ax audit localhost:3000 --tunnel ora
```
The hostname is an unguessable UUID under `*.t.agentfront.sh` that stops answering the moment the audit finishes (or you press Ctrl-C). Today the tunnel is opened in `public` mode, because ora's scanner cannot yet present a per-tunnel credential; a `protected` mode the scanner can use is on the roadmap.
**Bring your own tunnel.** Pass `--tunnel-cmd` (or set `ORA_TUNNEL_CMD`) with any command that exposes the local server and prints its public `https://` URL. The CLI runs it, audits the URL it prints, and tears the tunnel down when done:
```
ax audit localhost:3000 --tunnel-cmd 'ora tunnel 3000 --access public' # ora's tunnel CLI (npm i -g @ora-ai/cli; ora login)
ax audit localhost:3000 --tunnel-cmd 'ngrok http 3000 --log stdout'
```
- **The CLI ships no tunnel vendor and never downloads executables at runtime** — ora's own tunnel is first-party protocol code over Node's built-in WebSocket; any other tunnel tool you already have works through `--tunnel-cmd`, as long as it prints its public URL to stdout or stderr.
- With `ora tunnel`, `--access public` is required today: `protected` prints a tokenized link the scanner cannot use.
- Off-site checks (registry listings, brand search) usually fail for a throwaway tunnel hostname — the report says so. Use tunnel audits to iterate on your on-site surface, not to compare scores.
- Free tiers of some tunnel vendors serve an interstitial warning page to browser-like requests, which can distort what the scanner sees — prefer a vendor/plan that serves your origin directly.
## webmcp-audit
```
ax webmcp-audit [--chrome-endpoint e] [--min-score n] [--api-key k] [--json] [--show-passing]
```
```bash
ax webmcp-audit http://localhost:3000
```
The dev-loop counterpart to the audit at [webmcp.ora.ai](https://webmcp.ora.ai): check the
`document.modelContext` tools your page registers *before* you ship it, and get the same answer
you will get once it is public.
Any `http(s)` URL works — a dev server, a staging host, or a site you do not own. A page with no
WebMCP surface is not an error: it comes back `absent` with the checks that cannot apply marked
`Not applicable`, which is a perfectly good way to see what the audit looks like before you have
written any tools.
**It uses a Chrome you already have.** WebMCP tools only exist inside a live page, so the audit
opens yours in a real browser. It finds your installed Chrome, starts a headless one on a throwaway
profile, and stops it when the audit finishes — nothing to set up, and this package never downloads
or bundles a browser. Set `CHROME_PATH` if yours lives somewhere unusual.
To drive a browser you started yourself, pass `--chrome-endpoint` (a `ws://` URL, an http origin,
or `host:port`) and start it with `--remote-debugging-port`. The command never attaches to a
browser you did not name — including anything already on port 9222 — because silently borrowing a
Chrome with the WebMCP flags on would score your page higher here than it scores on ora.
**Do not enable the WebMCP flags on a Chrome you point it at.** The audit asks whether a real
browser gives an agent WebMCP on your page. A Chrome started with `--enable-features=WebMCPTesting`
answers yes for every site, so a page can come back `active` here and `testing-only` on ora. The
report tells you when the browser answered for the page; where that actually moved a check, ora
annotates that check itself. Keep a flag-enabled Chrome for
[`@ora-ai/webmcp-verify`](https://www.npmjs.com/package/@ora-ai/webmcp-verify), which needs the API
present in order to call your tools.
**What it measures, and where.** The capture is local: three passes over your entry page, the same
passes and the same probe code ora's worker runs, plus a viewport screenshot at ora's 1280×720. As
ora does, it loads the page with images, media and webfonts refused — stylesheets still apply — so
the screenshot the grader sees is the one ora would have taken. That capture is
posted to ora, which runs the 16 checks, two model hops and the tool-selection simulation, and
streams back a score. The CLI scores nothing itself (design decision #1) — grade, findings,
category scores and the simulation are all rendered as ora sent them.
**Availability is a gate, not a score.** Before anything is graded, ora decides whether an
in-browser agent can use the page at all. A page that cannot comes back `not agent-ready` with the
reasons why, and no score or grade — the reasons are the answer.
Past that gate the score is four pillars — `shared-experience` (30), `task-completion` (25),
`tool-quality` (25), `trust` (20). It is **not** the 0–100 agent-readiness score from `ax audit`:
different inputs, different rubric, never comparable.
A pillar can read 0 because it earned nothing or because nothing in it could be measured; the
report marks the second case rather than leaving you to guess.
**Two check statuses mean "no score" and are not the same thing.** `Not applicable` means the check
had nothing to measure and your page is not charged for it. `Not measured` means it applied and
could not be measured — that counts against evidence coverage, and below the coverage gate ora
withholds the letter grade while still reporting the score. A withheld grade beside a real score is
normal, not an error.
**What leaves your machine.** Scoring happens on ora's side, so the capture is uploaded: the page
URL, every tool your page registers (names, descriptions, JSON Schemas, annotations), the WebMCP
evidence found in your page's own scripts, and **a 1280×720 screenshot of the page**. Point it at a
logged-in or seeded dev environment and that screenshot shows whatever was on screen. Your source
code is never read and never sent.
**Nothing is stored.** A local audit is scored and returned and then forgotten. It never reaches
the leaderboard, the badge, or your public scorecard — to appear there, publish and run the audit
on webmcp.ora.ai. Pass `--json` and pipe it to a file if you want to see exactly what was sent
back; the capture itself is what the report's header describes.
Anonymous use is rate-limited to 10/min and 20 per 24h per IP; an ora API key
(`--api-key` / `ORA_API_KEY`) lifts that.
### CI gate
```yaml
- run: npx ax webmcp-audit http://localhost:3000 --min-score 70
```
Any CI image with Chrome installed works; nothing to launch first.
Exit codes match `audit`: `0` ok · `1` below `--min-score` · `2` usage (including no debuggable
Chrome) · `3` API. The gate reads the score, not the grade — a withheld grade would otherwise let
everything through.
## deep-journey
```
ax deep-journey [--intent id | --task text] [--agent id] [--api-key k] [--no-stream] [--json]
```
```
ax deep-journey stripe.com --intent pricing
ax deep-journey zapier.com --task "Find how to build a Slack → Sheets zap" # needs a partner key
```
Runs a real AI agent at a site through ora's **public** journey API — no key, no workspace needed for curated tasks. The agent's trajectory streams live as progress lines; the terminal output is ora's verdict, the billable step count, and the generated insight:
```
✓ task satisfied stripe.com
14 steps · 6 turns · 41.2s
The agent found the pricing page in two hops via llms.txt and correctly
summarised the standard and custom tiers.
- llms.txt was load-bearing: it linked /pricing directly
- no web search was needed; every path came from on-site links
For the 4-layer readiness audit, run: ax audit stripe.com
```
Anonymous callers run curated intents (`pricing`, `signup`, `api-docs`, `integrate`, `support`, …) — list them with `GET https://ora.ai/api/journey/intents`, and the accepted agents with `GET https://ora.ai/api/journey/agents`. The public caps are 100 runs per rolling 24h per target and 200 runs per 24h per IP (plus a 20/min burst limit); a target at its cap answers with the most recent stored run (`rate_limited: true`) instead of an error, so repeat CI invocations are cheap. Verdict and step count come from ora's contract as-is — the CLI never re-judges a run.
**Keyed tier:** an ora-issued partner API key (`--api-key`, or `ORA_PARTNER_API_KEY` / `ORA_SCAN_API_KEY` in the environment) unlocks `--task` — a free-text task of 4–300 characters that replaces the curated intent — and moves the caller to an allowance of 1000 runs per rolling 24h per key with no per-target cap and no burst guard. Keys are issued manually by ora (no self-serve signup). A wrong or unrecognized key with a *curated* intent silently degrades to the anonymous tier; `--task` without a recognized key is an error.
If the live stream goes quiet (no frames for 120s) the CLI does not abandon the run — it switches to polling the run detail until the server reports a terminal state.
| Flag | Effect |
|---|---|
| `--intent ` | Curated task id (server default when omitted) |
| `--task ` | Free-text task — needs a partner API key; mutually exclusive with `--intent` |
| `--api-key ` | ora partner API key; also read from `ORA_PARTNER_API_KEY` (then `ORA_SCAN_API_KEY`) |
| `--agent ` | Agent from the public roster (default: ora's pick) |
| `--no-stream` | Poll for the result instead of streaming the trajectory |
| `--json` | Print the terminal run detail (verdict, step_count, result) as JSON |
deep-journey exit codes: `0` run succeeded (any verdict) · `1` run failed · `2` usage error · `3` API unreachable / rate-limited.
This differs from `ax journey` (below), which drives the authenticated platform with free-text goals and renders the full trajectory graph; `deep-journey` is the zero-setup public path.
## skill
```
ax skill # list the registry
ax skill agent-ready-website # print a SKILL.md
ax skill agent-ready-website --install # write .claude/skills//SKILL.md
ax skill --json # the raw registry index
```
Skills come from ora's public registry (`https://ora.ai/.well-known/agent-skills/`) and every byte is verified against the registry's sha256 digest before it is printed or installed. Skill content is never bundled into this package. `--dir ` overrides the install directory, and `--json` prints the registry index as JSON instead of the formatted list.
## Library use
The same contract-typed client the CLI uses is importable:
```ts
import { audit } from "ax";
const { result } = await audit("stripe.com");
console.log(result.score, result.grade, result.topFixes);
```
`result` is the raw versioned audit payload (`AuditScanResult` / `AuditScoreResult`, generated from ora's OpenAPI spec). `fetchSkill` / `fetchSkillIndex` expose the digest-verified skill registry.
The public journey client is exported too — the same no-key path as `ax deep-journey`:
```ts
import { deepJourney, fetchJourneyAgents } from "ax";
const { agents, defaultId } = await fetchJourneyAgents();
const agent = agents.find((a) => a.id === defaultId)!;
const { detail } = await deepJourney("stripe.com", {
intentId: "pricing",
harness: agent.harness,
model: agent.model,
});
console.log(detail.verdict, detail.step_count, detail.result?.insight.summary);
```
`detail` is the contract's `JourneyRunDetail`; `fetchJourneyIntents` lists the curated intents. The same public caps apply — a capped target resolves with the most recent stored run (`cached: true` on the outcome), not an error.
## For ora partners
Everything above runs without an account. The commands and flags in this section are for teams working with ora directly: they need an ora-issued API key, and there is no self-serve signup — keys are issued by ora (contact [hello@ora.ai](mailto:hello@ora.ai)). A partner key also lifts the anonymous rate limits on `audit`, `webmcp-audit`, and `deep-journey` via their `--api-key` flags.
### journey
```
ax journey "" [--domain d] [--harness h] [--model m] [--json]
```
```
ax journey "Find the API docs and how to authenticate" --domain stripe.com
```
Triggers a real agent run on ora's platform and renders the journey live — the whole graph repaints in place as the agent moves:
```
▶ stripe.com · claude-code
9 steps · 3 reasoning · 1 search
╭────────────────────────────────────────────────────────────────────────────────╮
│ ◆ Find the API docs and how to authenticate. Fetch the actual pages to verify. │
╰─┬──────────────────────────────────────────────────────────────────────────────╯
│ (…) The user is asking me to find the API documentation for Stripe…
│ ╭──────────────────────────────────────────────╮
╰──┤ 🔍 "Stripe API documentation authentication" │
╰─┬────────────────────────────────────────────╯
│ ╭─────────────────────────────────────────╮
├──┤ 🌐 docs.stripe.com/api/authentication ✓ │
│ ╰─────────────────────────────────────────╯
│ ╭───────────────────────────╮
╰──┤ 🌐 docs.stripe.com/apis ✓ │
╰───────────────────────────╯
────────────────────────────────────────────────────
✓ completed 9 turns · 30.8s · 119.1k tokens · $0.031
Visibility █████████████████████░ 95/100 ✓ intent satisfied
```
Edges come from ora's own per-step attribution (link-follows nest under the page that linked them; searches hang off the intent). `(…)` markers are the agent's reasoning, placed above the action it led to.
| Flag | Effect |
|---|---|
| `--domain ` | Site the agent targets (e.g. `stripe.com`) |
| `--harness ` | Agent harness: `claude-code` (default), `codex`, `openclaw`, `hermess`, `claude-agent`, `eve`, … |
| `--model ` | Model override (e.g. `claude-haiku-4-5-20251001` — cheap and good for demos) |
| `--json` | Full result as JSON: run metadata, usage, insight, and the raw trajectory |
Note: run-to-run variance is real — agents sometimes answer from search snippets without fetching, which scores differently. Run a few journeys before drawing conclusions.
Journey exit codes: `0` run outcome `success` · `1` non-success outcome · `2` any error.
#### Setup
`journey` needs an ora platform API key (scopes `runs:read` + `runs:write`):
```sh
export ORA_API_KEY=ora_sk_... # your shell, CI secret store, etc.
ax journey "Find the pricing page" --domain example.com
```
For local development, copy `.env.example` to `.env` — the CLI reads a `.env` in the working directory on startup (exported variables win over the file).
## Environment variables
All configuration is environment-first: the consumer defines the variables, the CLI only reads them.
A `.env` in the working directory is read on startup — copy `.env.example` and fill it in. Anything already exported wins over the file, so a key set for your shell or by CI beats a stale `.env`.
| Var | Used by | Default | Purpose |
|---|---|---|---|
| `ORA_API_URL` | audit, deep-journey, skill | `https://ora.ai` | Public API base (no auth) |
| `ORA_PLATFORM_URL` | journey, audit `--tunnel ora` | `https://api.agentfront.sh` | Authenticated platform API base |
| `ORA_TUNNEL` | audit | — (optional) | `ora` opens ora's own tunnel for a local target (same as `--tunnel ora`) |
| `ORA_TUNNEL_CMD` | audit | — (optional) | Tunnel command for a local target (same as `--tunnel-cmd`); wins over `ORA_TUNNEL` |
| `ORA_API_URL` | webmcp-audit | `https://ora.ai` | Also the ingest base for `webmcp-audit` |
| `ORA_API_KEY` | journey, audit `--tunnel ora`, webmcp-audit | — (required for journey and `--tunnel ora`) | Secret key (`ora_sk_…`); journey and `--tunnel ora` exchange it for a bearer token (the tunnel needs the `tunnels:write` + `tunnels:connect` scopes), webmcp-audit sends it to lift the ingest rate limits |
| `ORA_SCAN_API_KEY` | audit, deep-journey | — (optional) | ora-issued scan API key; lifts every scan rate limit (issued manually by ora). deep-journey accepts it as a partner-key fallback |
| `ORA_PARTNER_API_KEY` | deep-journey | — (optional) | ora-issued partner API key; unlocks `--task` and the 1000/24h keyed allowance |
## Contract versioning
The audit types are generated from ora's served OpenAPI spec (`pnpm contract:gen`); the build is pinned to a contract version and CI fails when production drifts from the checked-in types. At runtime, the CLI prints one stderr warning when ora reports a newer contract than the build — upgrade with `npm i -g ax@latest`.
## Migrating from 0.1.x
- `ax scan` is now `ax audit` (no alias).
- `--json` prints the raw ora audit payload instead of the old reshaped envelope: stable check ids, layer ids, `grade`, `topFixes`, and `contractVersion` — key your scripts to those fields.
- Audit API failures now exit `3` (usage errors stay `2`).
## Development
```sh
pnpm install
pnpm test # vitest (watch)
pnpm test:ci # single run + coverage
pnpm typecheck
pnpm lint
pnpm contract:gen # regenerate src/contract/ from the served OpenAPI spec
pnpm build # tsup → dist/main.cjs (bin) + dist/index.* (library)
pnpm smoke # run the built binary the way a user would (needs build)
pnpm smoke:webmcp # webmcp-audit end to end: real Chrome + mock ingest (needs build)
pnpm verify:pack # check the tarball npm would publish (needs build)
node dist/main.cjs audit https://example.com
```
`webmcp-audit` is the one command whose core cannot be unit-tested — it needs a browser. Two
commands cover it, and CI runs both on every PR:
```sh
AX_WEBMCP_LIVE=1 pnpm test:capture # the three capture passes against a real Chrome
pnpm build && pnpm smoke:webmcp # the whole chain: browser → capture → ingest → report
```
Both launch their own headless Chrome and stop it again. In a container where Chrome's sandbox
cannot start, set `AX_WEBMCP_NO_SANDBOX=1` — never on a development machine, since the page being
audited is untrusted and the sandbox is what contains it.
Manual testing without burning the live rate limits:
```sh
node scripts/mock-scan-server.mjs &
ORA_API_URL=http://localhost:8799 node dist/main.cjs audit https://example.com
```
```sh
node scripts/mock-webmcp-ingest.mjs &
ORA_API_URL=http://localhost:8798 node dist/main.cjs webmcp-audit http://localhost:3000
```
The webmcp mock replays the checked-in real ingest fixture, and takes a failure mode as its second
argument — `shim`, `rate`, `scoring` or `stream` — so every error branch is reachable without the
live endpoint.
To exercise the published-package experience locally: `npm pack`, then `npx ./ax-.tgz audit https://example.com`, or `pnpm link --global` and use `ax` directly.
## Releasing
Publishing runs from GitHub Actions only. Every push to `main` updates a
release-please PR; merging it tags, creates the GitHub Release, and publishes
to npm with provenance. See [RELEASING.md](RELEASING.md) for the runbook,
required setup, and failure recovery.
## License
MIT