# Closing the REST API gaps in `@x12i/api-simulator`

> Status: engine Phases 1–3 and Playground/Studio v1 are implemented in-tree.
> See also [`playground-studio.md`](./playground-studio.md) and
> [`package-simulators.md`](./package-simulators.md) (composing package REST with
> `@x12i/static-memorix` stays out of the core engine).

This is a plan for taking the simulator from "handles JSON REST CRUD APIs well" to "can credibly stand in for almost any REST API" — scoped deliberately to REST over HTTP. gRPC, WebSockets, GraphQL, and streaming/SSE responses are explicitly out of scope for this pass; they're different enough protocols that folding them in now would blur the core's current clarity for uncertain payoff. Everything below assumes the target is still request-in, response-out HTTP, just with a much wider range of what "request" and "response" can look like.

## Guiding principles

Two things made the existing engine easy to reason about and worth preserving as we extend it: the transport-agnostic core (`dispatch()`/`tryDispatch()` never touch Node's `http` module directly, so anything built on top of them is portable) and the package's zero-runtime-dependency stance (only `typescript` and `@types/node` are even devDependencies today). Both should survive this work. Concretely, that means every gap gets closed either as a new optional module behind its own subpath export (matching the existing `./node` pattern), as new optional fields on existing types that default to today's behavior when omitted, or as an extension point — a place to plug in your own parser, validator, or auth scheme — rather than a bundled dependency for something like multipart parsing or JSON Schema validation. If a feature can only be done well by adding a real dependency (a YAML parser for OpenAPI specs, a multipart parser, a JSON Schema engine), the default path stays dependency-free and the dependency becomes something the consumer brings themselves through that extension point. Every new module also gets its own `tests/*.test.mjs` (so it's covered by the existing `npm run validate`) and its own README section, and where it fits naturally, an extension of the library-demo tutorial so the new capability has the same kind of real, runnable demonstration the existing behaviors do.

## Phase 1 — quick, additive, low-risk

These don't touch any existing type signatures and can land independently of each other.

The request-body parser in the Node adapter currently forces any non-JSON body through `toString('utf8')`, which quietly corrupts real binary uploads. The fix is small: only stringify when the content type actually looks like text (`text/*`, `application/json`, `application/x-www-form-urlencoded`, `+json`/`+xml` suffixes), and hand back the raw `Buffer` otherwise so a simulation handler can inspect length, magic bytes, or forward it untouched. This alone closes the "binary request bodies get mangled" gap with almost no risk.

The equivalent gap on the response side is more visible: `writeJson` always calls `JSON.stringify(body)`, so a `simulation` handler that returns the string `"hello"` comes back over the wire as `"\"hello\""`. The fix is a small `RawResponseBody` wrapper — `new RawResponseBody(content, contentType?)` — that `writeJson` recognizes and writes through verbatim (string or `Buffer`, with the given or inferred content type) instead of stringifying. Everything that doesn't use the wrapper behaves exactly as it does today, so this is fully additive. Together with the Buffer fix above, this is what actually lets the simulator stand in for an endpoint that returns HTML, XML, CSV, or a binary download, not just JSON.

The biggest practical gap for REST specifically is the lack of any reusable stateful store — every project currently has to hand-roll its own `Map`-based ledger the way the library demo's `checkout-ledger.ts` does. A new `@x12i/api-simulator/store` module generalizes that pattern into a small `createStore(seed, options)` helper with the CRUD operations a REST resource actually needs (list, get, create, update, remove, and reset for test isolation between runs), plus basic filtering and pagination helpers. This doesn't change how `data` works on an `ApiDefinition` at all — it's a separate, opt-in utility that `simulation` handlers import and use, the same way `checkout-ledger.ts` already does today, just no longer reinvented per project.

Auth and rate-limiting are too varied to bake into the engine as one true implementation, so instead of a specific auth scheme, a `@x12i/api-simulator/helpers` module ships small composable functions — `requireHeader`, `requireBearerToken`, `rateLimit`, and similar — each a higher-order function that wraps a `SimulationFunction` and returns a new one. This keeps the "simulation handlers are just functions" philosophy intact while giving people something to reach for instead of writing the same 401/429 boilerplate in every project.

Finally, two smaller operational additions round out Phase 1: an opt-in `recordHistory` option on `createApiSimulator` that keeps a bounded ring buffer of recent dispatches (so a test can ask "what was actually sent to this endpoint" the way `nock` or MSW let you assert on calls), and an opt-in `validateTemplates` startup check that renders every `relative` endpoint's template against its API's `data` alone, catching an obviously broken `{{data.typo}}` reference at construction time instead of on first request. Neither can fully replace request-time validation, but both catch real mistakes earlier for very little cost.

## Phase 2 — moderate design work, still additive

Multipart form data and urlencoded bodies are common enough in REST APIs (file uploads, classic HTML forms) that they deserve more than "read it as a Buffer and figure it out yourself." The approach is a pluggable body-parser extension point on the Node adapter — a `bodyParsers` option keyed by content type — with built-in parsers for JSON, urlencoded, and plain text shipped by default, and multipart deliberately left as a documented recipe ("here's how to plug in `busboy` or similar") rather than a bundled dependency. This keeps the zero-dependency default intact while still closing the gap for anyone who needs it.

Being able to point the simulator at a real OpenAPI spec and get a starting skeleton, instead of hand-authoring every endpoint, is probably the single highest-leverage addition for anyone mocking a third-party API they didn't design. A new `@x12i/api-simulator/openapi` module takes a parsed OpenAPI 3.x document and generates `EndpointDefinition[]` skeletons — path, method, and a `fixed` behavior seeded from the spec's examples or schema defaults — that a person then edits into real `relative` or `simulation` behaviors where it matters. Scope this to JSON specs first; YAML specs are common in the wild, but rather than bundling a YAML parser, the importer should just accept an already-parsed JS object, so anyone with a YAML spec pipes it through `yaml.parse()` (or whatever they already use) before handing it in.

CORS is a REST-specific enough concern to be worth a dedicated, opt-in feature rather than something everyone reimplements by hand: a new optional `cors` field on `ApiDefinition` (allowed origins, methods, headers) that `createApiSimulator` expands into synthetic `OPTIONS` endpoints and CORS response headers on real matches, only when the field is present.

On the adapter side, the highest-value addition is a `@x12i/api-simulator/fetch` module exposing `createFetchHandler(simulator, options)` — a `(request: Request) => Promise<Response>` function built on the same WHATWG `Request`/`Response` types that Cloudflare Workers, Deno, Bun, and Next.js route handlers already speak natively. That one adapter, plus documented one-line recipes for wrapping `dispatch()`/`tryDispatch()` directly in Express or Fastify, covers far more ground than writing and maintaining a bespoke adapter per framework.

## Phase 3 — bigger changes, worth a design review before committing

A few items need more care because they touch existing exported types rather than adding new ones. Multi-value response headers (and therefore proper `Set-Cookie` support) mean widening `Headers` from `Record<string, string>` to `Record<string, string | string[]>` — Node's `http` module already accepts array header values natively, so the runtime change is small, but it touches `engine.ts`'s header normalization and `node.ts`'s request/response handling, and it's worth a deliberate compatibility pass (and probably its own changelog callout) rather than folding it in quietly.

Host- or subdomain-based routing (useful for simulating multi-tenant APIs distinguished by hostname rather than path) is a contained addition — an optional `host` matcher on `ApiDefinition` checked alongside method and path in `tryDispatch` — but it's lower priority until there's a concrete need for it.

Request/response schema validation against the endpoint's contract is valuable but shouldn't force a specific validator on everyone; like the body-parser extension point, this should be a `validators` hook that accepts whatever schema-checking function (ajv, zod's `safeParse`, a hand-rolled one) the consumer already uses, called with the matched request and the endpoint definition.

Advanced path matching — regex-constrained parameters (`:id(\d+)`) or optional segments (`/things/:id?`) — is a self-contained change inside `path-matcher.ts` and low risk, but also low urgency; it's worth doing opportunistically rather than scheduling.

Record-and-replay (hit a real API once, capture the traffic, emit ready-to-edit `EndpointDefinition`s from it) is powerful but is really a companion dev-time tool rather than something the runtime library needs to carry — it's worth prototyping as a separate script or small CLI rather than bundling it into the core package's surface area.

## What stays out of scope for now

Streaming and chunked responses (SSE, token-by-token completions, long-polling), gRPC/protobuf, WebSockets, and GraphQL-specific routing are all real "API" shapes, but they're different enough problems that reaching for them now — before Phase 1 and 2 are even in place — would spend effort on breadth the REST-focused use case doesn't need yet. Worth revisiting if a concrete project needs one of them; not worth speculatively building today.

## Roadmap at a glance

| Phase | Addition | New surface | Notes |
|---|---|---|---|
| 1 | Binary-safe request body fallback | change inside `node.ts` | no API change |
| 1 | Raw / non-JSON response passthrough | `RawResponseBody` wrapper | additive |
| 1 | Generic in-memory CRUD store | `@x12i/api-simulator/store` | additive |
| 1 | Auth / rate-limit helper functions | `@x12i/api-simulator/helpers` | additive |
| 1 | Request history + startup template check | new options on `createApiSimulator` | opt-in |
| 2 | Pluggable body parsers (urlencoded, text, multipart recipe) | `bodyParsers` option | additive |
| 2 | OpenAPI-to-skeleton importer | `@x12i/api-simulator/openapi` | additive, JSON specs first |
| 2 | CORS / OPTIONS auto-expansion | `cors` field on `ApiDefinition` | additive |
| 2 | Fetch-standard adapter | `@x12i/api-simulator/fetch` | additive |
| 3 | Multi-value headers / cookies | `Headers` type widened | needs compatibility review |
| 3 | Host / tenant-based routing | `host` field on `ApiDefinition` | additive, low urgency |
| 3 | Pluggable schema validation | `validators` hook | additive |
| 3 | Advanced path patterns | change inside `path-matcher.ts` | additive, low urgency |
| 3 | Record-and-replay | separate companion tool | exploratory |

## Where to start

The best return for the least risk is the pairing of the raw-response wrapper and the generic store — together they close the two gaps that show up in almost every real REST mock (non-JSON responses and state that actually changes across requests), and neither touches an existing type signature. That's the natural place to start whenever you want to move from planning to building.