zpi-sdk — Universal TypeScript SDK
for the Zapi (Zest API) Scraper Platform

zpi-sdk is a universal, zero-dependency, API-key-first TypeScript client for the Zapi (Zest API) scraper platform. Construct a ZpiClient, then call any scraper end-to-end — single call, streaming, or bulk — with full types and a structured error hierarchy. Everything runs off a single injectable fetch seam, so the same build works on Node, Bun, Deno, and the browser.

[Quick start](#quick-start) • [Why zpi-sdk](#why-zpi-sdk) • [Install](#install) • [What you can build](#what-you-can-build) • [Errors](#error-handling) • [Runtimes](#runtime-support) • [Docs](https://zpi.web.id/docs)

> [!NOTE] > This README is a **high-level overview**. The complete API reference, guides, and recipes live in the documentation site at ****. --- ## Quick start Grab an API key from [zpi.web.id](https://zpi.web.id), construct a client, and call any scraper — `run()` returns the unwrapped result data directly: ```typescript import { ZpiClient } from 'zpi-sdk' const client = new ZpiClient({ apiKey: 'zpi_...' }) const data = await client.run('social:instagram', 'profile', { username: 'instagram' }) console.log(data) ``` That is the whole integration — **no HTTP method to pick, no path templates to learn**: - The SDK sends the right verb (GET/POST) on the first request and remembers it per endpoint, process-wide. Pass codegen's table to `client.useMethods(ZPI_METHODS)` and it never has to ask. - Endpoints with path params like `resolve/:url`? Just put `url` in the params object — done. `run('bypass-tools:encurtador', 'resolve', { url: '...' })` and `run('bypass-tools:encurtador', 'resolve/:url', { url: '...' })` both work. ## Build with AI zpi-sdk ships a built-in **MCP client** behind the `zpi-sdk/mcp` subpath, so AI agents can discover and call every scraper as MCP tools. It speaks JSON-RPC over the remote `/mcp` Streamable-HTTP server — hand-rolled, so it stays **zero-dependency**: ```typescript import { createMcpClient } from 'zpi-sdk/mcp' const mcp = createMcpClient({ apiKey: 'zpi_...' }) const tools = await mcp.listTools() // handshake runs lazily on first call const result = await mcp.callTool('run_scraper', { /* tool args */ }) ``` The `mcp` module never loads from the root entry, so the core stays lean. See the full guide → **[zpi.web.id/docs](https://zpi.web.id/docs)**. ## Why zpi-sdk - **Zero dependencies** — no runtime deps, no node builtins; one injectable `fetch` seam powers every runtime from Node to the browser. - **One method, every scraper** — `client.run('social:instagram', 'profile', params)` covers the whole catalog; no per-scraper wrappers to learn. - **Zero ceremony** — no `{ method: 'GET' }` guessing (auto-detected + memoized per endpoint) and path params (`/:id`, `/:url`) are just regular fields in `params`. - **Typed error hierarchy** — every failure is a `ZpiError` subclass (`ZpiRateLimitError`, `ZpiPlanGateError`, …) with typed fields like `retryAfterSec` and `upgradeUrl`. No status-code guessing. - **Streaming out of the box** — `client.stream(…)` returns an async iterable of SSE events, reading via `body.getReader()` so it streams incrementally everywhere. - **Bulk jobs** — submit many items, `job.wait()` with progress callbacks; submits auto-reuse an `Idempotency-Key`, so retries never create duplicate jobs. - **Public catalog discovery** — list scrapers, categories, endpoint schemas, and stats without auth. - **Webhook verification built in** — `zpi-sdk/webhooks` verifies `X-Zpi-Signature` (HMAC-SHA256, timing-safe) and returns typed events. - **Typed codegen** — `npx zpi codegen --scan .` reads your source, finds the scrapers you actually call, and generates bindings for just those; `run()` then returns a typed result with no annotation at the call site. - **Safe retries** — only network errors and `429/502/503/504` are retried (exponential backoff + jitter); a POST is never blind-retried without an `idempotencyKey`. API key redacted from every error and log. ## Install ```bash npm i zpi-sdk # or: pnpm add zpi-sdk • yarn add zpi-sdk • bun add zpi-sdk ``` Requires **Node.js v20+**. Deno needs no install step — import via the `npm:` specifier: ```typescript import { ZpiClient } from 'npm:zpi-sdk' ``` Zero runtime dependencies, dual ESM/CJS with `.d.ts` + `.d.cts` types, and a `bin` (`zpi`) for codegen. All configuration is optional beyond the API key: ```typescript const client = new ZpiClient({ apiKey: 'zpi_...', baseURL: 'https://api.zpi.web.id', // override the default base URL timeoutMs: 30_000, // per-request timeout maxRetries: 2, // retry budget for retryable failures fetch: globalThis.fetch, // inject your own fetch (tests, proxies, edge runtimes) }) ``` ## What you can build ### Run any scraper ```typescript const profile = await client.run('social:instagram', 'profile', { username: 'instagram' }) const video = await client.run('downloader:tiktok', 'video', { url: 'https://tiktok.com/...' }) const gold = await client.run('finance:goldprice', 'latest', {}) ``` ### Streaming For chunked / SSE endpoints, iterate the async iterable returned by `client.stream(...)`: ```typescript for await (const event of client.stream('ai:chat', 'completions', { prompt: 'hi' })) { // StreamEvent = SseEvent ({ event?, data, id? }) | string console.log(typeof event === 'string' ? event : event.data) } ``` ### Bulk jobs Submit many items at once, then `wait()` for completion with optional progress reporting: ```typescript const job = await client.bulk.submit('social:instagram', 'profile', [ { url: 'https://instagram.com/a' }, { url: 'https://instagram.com/b' }, ]) const result = await job.wait({ onProgress: (j) => console.log(j.succeeded, '/', j.total), }) for (const item of result.items ?? []) { console.log(item.status, item.data ?? item.error) } ``` ### Catalog discovery (public, no auth) ```typescript const { items } = await client.catalog.list({ cat: 'social', limit: 20 }) const detail = await client.catalog.get('social:instagram') const schema = await client.catalog.schema('social:instagram', 'profile') const stats = await client.catalog.stats('social:instagram') ``` ### Handle webhooks Verify and parse incoming webhook deliveries (`bulk.completed`, `quota.warning`, `request.error`, …) with the zero-dependency `zpi-sdk/webhooks` helper — Web Crypto HMAC with a timing-safe compare, so it runs on Node, Bun, Deno, and edge workers: ```typescript import { parseWebhook } from 'zpi-sdk/webhooks' // In your HTTP handler — pass the RAW body string, not the parsed JSON: const event = await parseWebhook(rawBody, { signature: req.headers['x-zpi-signature'], secret: process.env.ZPI_WEBHOOK_SECRET, }) // → { id, event: 'bulk.completed' | …, data, deliveredAt } // Throws ZpiWebhookVerifyError on a bad signature or malformed payload. ``` ### Typed codegen Generate per-scraper bindings from the **live** catalog — `run()`'s params and result become fully typed, with no annotation at the call site: ```bash npx zpi codegen --scan . # only the scrapers your code calls → ./zpi-sdk.gen.d.ts npx zpi codegen # the whole catalog npx zpi codegen --out ./types/zpi.gen.d.ts --filter social ``` `--scan ` reads your source for `run("category:scraper", "endpoint")` calls and emits only those. On a 500-endpoint catalog that is the difference between a 160 KB declaration file and a 600-byte one. Calls it cannot resolve statically — a variable endpoint, a computed key — are simply not emitted, and any scraper key you call that the catalog does not know is reported: ``` zpi codegen: wrote ./zpi-sdk.gen.d.ts (1 scrapers, 2 endpoints) zpi codegen: called but not in the catalog — typo:scraper ``` The output is **types-only** (zero runtime import); regenerate any time the catalog drifts. The generated header records the exact command used, so a `--scan` file is never regenerated as the full catalog by accident. ## Error handling Every failure throws a `ZpiError` (or a subclass) carrying `status`, `code?`, `raw`, and `requestId?` — catch and branch on the class: ```typescript import { ZpiClient, ZpiPlanGateError, ZpiRateLimitError, ZpiError } from 'zpi-sdk' try { await client.run('social:instagram', 'profile', { username: 'instagram' }) } catch (e) { if (e instanceof ZpiPlanGateError) console.log('upgrade required:', e.requiredPlan, e.upgradeUrl) else if (e instanceof ZpiRateLimitError) console.log('retry after', e.retryAfterSec, 's') else if (e instanceof ZpiError) console.log('zpi error:', e.status, e.code, e.raw) } ``` | Class | When | Notable fields | | --- | --- | --- | | `ZpiAuthError` | Missing / invalid API key (401) | — | | `ZpiPlanGateError` | Endpoint requires a higher plan (403) | `requiredPlan`, `upgradeUrl` | | `ZpiRateLimitError` | Rate limit hit (429) | `limit`, `used`, `window`, `retryAfterSec` | | `ZpiInvalidParamsError` | Params failed validation (400/422) | `errors[]` | | `ZpiExecError` | The scraper ran but failed | `error`, `errors`, `context` | | `ZpiNetworkError` / `ZpiTimeoutError` / `ZpiAbortError` | Transport failure / timeout / abort | `cause` | | `ZpiMcpError` | JSON-RPC error from `/mcp` | `code`, `data` | > The full taxonomy (bulk, idempotency, 404/405/5xx classes) is documented at **[zpi.web.id/docs](https://zpi.web.id/docs)**. ## Runtime support zpi-sdk ships dual ESM/CJS entry points with type declarations for both module systems, and is verified to load on: | Runtime | ESM | CJS | | ------- | --- | --- | | Node.js `>=20` | ✅ | ✅ | | Bun | ✅ | ✅ | | Deno (`npm:` specifier) | ✅ | ✅ | | Browser | ✅ | — | Package managers: **npm**, **pnpm**, **yarn**, and **bun** are all supported. ## Documentation - 🌐 [**zpi.web.id/docs**](https://zpi.web.id/docs) — full documentation site: guides, API reference, recipes - 🧭 [**zpi.web.id/apis**](https://zpi.web.id/apis) — browse the live scraper catalog - 🤖 [**MCP client**](https://zpi.web.id/docs) — wire every scraper into your AI agent as MCP tools - 📝 [**Changesets**](./.changeset) — pending release notes ## Issues & feedback Hit a problem or have a feature request? Open an [issue](https://github.com/zeative/zpi-sdk/issues). - [Buy me a coffee ☕](https://saweria.co/zaadevofc) • [Ko-Fi](https://ko-fi.com/zaadevofc) • [Trakteer](https://trakteer.id/zaadevofc) - ⭐ Star the repo on GitHub ## License Distributed under the **MIT License**. See [`LICENSE`](https://github.com/zeative/zpi-sdk/blob/main/LICENSE) for details.

zpi-sdk — Universal TypeScript SDK for the Zapi (Zest API) Scraper Platform

zpi-sdk — Universal TypeScript SDK
for the Zapi (Zest API) Scraper Platform