# API Test Strategy

**Purpose:** Define Layer 1/2/3 coverage rules for API testing — field-type technique routing, surface derivation, error coverage, dedup extensions, SLA baselines, and scope-conflict resolution. Specific to API interface testing; does NOT cover UI/functional test design (→ `test-design.md`).

**Used by:** `/tas-apitest-plan` Steps 3–7

**Depends on:**
- Layer 2 technique canonical counts → `istqb-techniques.md § Layer 2`
- Security technique routing → `security-test-matrix.md` (Layer 1 PE-testable vs Layer 2 tool-required)
- Error technique routing → `error-handling-strategy.md`
- TC ID format → `test-id-convention.md §3`

---

## Layer 1 — Scope Boundaries

### 1.0 Scope Source Determination (read-first gate)

Before enumerating any endpoint, determine `spec_baseline` — the authoritative source for `spec_endpoints`. Read ALL sources first; determine scope second.

**Source priority (highest wins — stop at first hit):**

| Priority | Source | When available |
|---|---|---|
| **1 (highest)** | TestChecklist §2.1 (`Stack=API` OR `Category=SEC` rows) | Feature-linked run AND TestChecklist file exists |
| **2** | Feature-Technical `§API Spec` section | Feature-linked run, no TestChecklist |
| **3** | OpenAPI file (JSON/YAML) | API Spec file input |
| **4 (lowest)** | Code glob only | No spec at any priority above |

```
spec_baseline = first non-empty source per priority table above
spec_endpoints = endpoints enumerated FROM spec_baseline only
```

**Read-first mandate:** TestChecklist (§2.1, §3, §5) MUST be read before `spec_baseline` is resolved. Store as digests (`inherited_tps`, `bva_constraints`, `test_data_catalog`, `inherited_risks`) — do NOT re-read later.

**If Priority 4 (code-only):** log `GAP-SPEC: No spec source found — code glob used as baseline. All endpoints are orphan candidates.` to `## Risk & Coverage Notes`.

### 1.1 Endpoint Inventory (L1 completeness gate)

After `spec_baseline` is resolved per §1.0, enumerate `spec_endpoints`:

```
FROM spec_baseline (per §1.0):
  Collect: HTTP method, path, path/query params, request body schema,
           response schemas per status code, auth requirement, tags

endpoint_inventory = deduplicated list of (method, path) pairs from spec_endpoints
```

**L1 blocking gate:** `endpoint_inventory` must be complete and confirmed before Phase 2 (Design) proceeds. If spec incomplete → ASK user, do not assume.

### 1.2 Contract ↔ Codebase Anchor Check

If API code path is available:
```
glob_endpoints = routes detected from controller/handler files
orphan_endpoints = glob_endpoints − spec_endpoints

IF orphan_endpoints ≠ ∅:
  For each orphan: ask [A] Out-of-scope  [B] Add to scope  [C] Defer
  scope_final = spec_endpoints ∪ user-confirmed-B-orphans
ELSE:
  scope_final = spec_endpoints
```

**Security note:** Endpoint implemented in code but excluded from spec = untested attack surface. Require explicit justification for any [A] choice.

### 1.3 Scope-Conflict Resolution (functest ↔ apitest)

Detect overlap with existing FuncTest spec for the same Feature:
```
glob {FEAT}-FuncTest.md → extract endpoint/AC claims
overlap = apitest_endpoints ∩ functest_endpoints

IF overlap ≠ ∅ → BLOCK, ask per overlapping endpoint:
  1. API Testing Only  (API contract/error handling)
  2. FuncTest Only     (UI integration, GUI-verifiable)
  3. Both (separate concerns): FuncTest=UI flow; ApiTest=contract+error
```

Document resolution in `## Risk & Coverage Notes`. `scope_final` excludes option-2 endpoints (FuncTest Only) from apitest scope.

### 1.4 Section Scope (4 test types)

| Section | Scope | L1 input source |
|---|---|---|
| **§1 Contract** | Schema, status codes, headers, auth, error codes per endpoint | API Spec |
| **§2 Integration** | Cross-service calls, external dependencies, async flows | Feature-Technical `## Integration Points` |
| **§3 Backward-Compat** | Breaking changes: removed/renamed fields, changed types, deprecated paths; additive changes (new optional field/endpoint) | API changelog or Feature-Technical §Versioning |
| **§4 NFR** | Perf SLA thresholds, OWASP API Top 10, pentest recon scenarios | `test-design.md § Platform NFR Baselines` + `security-test-matrix.md` (Layer 2 tool-required → log to `## Risk & Coverage Notes`) |

### 1.5 Breaking-Change Detection (§3 trigger)

Before designing §3 Backward-Compat TCs, scan Feature-Technical §Versioning or API changelog for change type:

| Change type | Classification | Min TCs + assertion |
|---|---|---|
| New **optional** field added to response | Additive — backward-compat | ≥1 TC: old client receives response, assert no parse error; new field present |
| New **optional** field added to request | Additive — backward-compat | ≥1 TC: old client omits field, assert 2xx (default applied) |
| New **endpoint** added | Additive — backward-compat | ≥1 TC: confirm endpoint reachable; no §3 regression test needed |
| Field **removed** or **renamed** | **Breaking** | ≥1 TC: old client sends removed/renamed field → assert documented error (not silent drop); ≥1 TC: response no longer contains field → assert old client handles gracefully |
| Field **type changed** (e.g. string → integer) | **Breaking** | ≥1 TC: old client sends original type → assert 400/422 with clear message |
| Endpoint **path changed** or **deprecated** | **Breaking** | ≥1 TC: old path → 404 or 301/308 redirect per spec; ≥1 TC via new path → 2xx |
| No §Versioning section in Feature-Technical | Gap | Log `GAP-SPEC: §Versioning section absent — §3 TCs skipped` to `## Risk & Coverage Notes` |

**BLOCKING:** If breaking change detected → §3 TCs are mandatory, not optional.

### 1.6 NFR Scope Gate (L1 — before scope_final is confirmed)

Determine which NFR categories are in scope **before** `scope_final` is locked. Run this immediately after §1.5 (or after §1.1 if no versioning section).

| NFR category | Postman-testable? | Action |
|---|---|---|
| **Performance** | ✅ Yes — if SLA documented | Include in `scope_final §4`; technique → BVA on response time (§3.7) |
| **Performance** | ❌ No SLA documented | Exclude; log `SKIP-PERF` to Risk & Coverage Notes |
| **Security — Layer 1** (XSS, SQLi, BOLA, auth bypass) | ✅ Yes — Postman scripted | Include in `scope_final §4`; surface routing → §2.4 |
| **Security — Layer 2 DAST** (OWASP ZAP scan) | ❌ No — tool-required | Exclude; log `SKIP-DAST` to Risk & Coverage Notes |
| **Pentest recon** | ⚠️ Partial — Postman as tool-assist | Include as stub-only TCs; Autoable = NO |
| **Backward-compat** | ✅ Yes — if breaking change detected in §1.5 | Include in `scope_final §3`; mandatory if breaking |

**Output:** `nfr_scope` flag set per category (IN / OUT / STUB) — used by §3.7 to determine TC generation. Do NOT generate NFR TCs for OUT categories.

---

## Layer 2 — Technique Selection

### 2.1 Field-Type → Technique Routing

For each field in request body / query / path:

| Field Type | Technique | Reference |
|---|---|---|
| Numeric with min/max | BVA | `istqb-techniques.md § BVA` |
| String with length constraint | BVA on length | `istqb-techniques.md § BVA` |
| Enum / categorical | EP (1 valid partition + 1 invalid per class) | `istqb-techniques.md § EP` |
| Date / date-range | BVA on boundary dates | `istqb-techniques.md § BVA` |
| Boolean flag controlling reversible action | 2 Happy TCs (one per state: enable + disable) | — |
| Object with mutually exclusive field groups | 1 Happy TC per mode (multi-mode rule) | — |
| Required vs optional | EP: present + absent | `istqb-techniques.md § EP` |
| Nested object / array | EP: empty, 1 item, max items | `istqb-techniques.md § EP` |

### 2.2 API-Specific Routing Rules

**State machine detection (BLOCKING if present):**
- Path param `{token}`, `{secret}`, `{key}`, `{code}` → add State Transition TCs
- Feature describes multi-step workflow → add State Transition TCs
- Otherwise → warning only, proceed

**Pagination rule:** Endpoints with `page`/`limit`/`cursor` params → EP: first page, last page, empty result, beyond-last-page.

**Full-payload rule:** If endpoint body spans fields covering multiple ACs → generate 1 additional Happy TC with all relevant fields populated.

**Idempotency rule:**
- PUT endpoints → 1 TC: send identical request twice, assert second response = first response and resource state unchanged
- DELETE endpoints → 1 TC: delete same resource twice, assert second call returns 404 (not 500)
- POST with `Idempotency-Key` header → 1 TC: replay same key, assert no duplicate resource created (see `istqb-techniques.md § Error Guessing`)

**Concurrency rule (BLOCKING if uniqueness constraint exists):**
- Unique constraint on any field → 1 TC: concurrent duplicate POST, assert exactly one 201 + one 409
- Shared mutable resource (balance, quota, seat) → 1 TC: concurrent PATCH, assert no lost-update (final state = sum of both writes or last-writer-wins per spec)
- If optimistic locking (`ETag` / `version` field) documented → 1 TC: stale ETag on PUT → 409 Conflict

### 2.3 Error Code Technique Routing

Error-code → technique → TC-count routing is owned by **`error-handling-strategy.md § Error Code → Technique Routing`** (incl. 401 3-partition, 403 IDOR, 404, 422 business-rule, 429 BVA). Apply it directly — do not re-table here.

### 2.4 Security Technique Selection

Attack-surface classification, techniques, and Autoable status are owned by **`security-test-matrix.md`** (Layer 1 manual-testable, Layer 2 tool-required, Layer 3 Autoable). This section owns only the **API-shape → which-surface trigger** (when to include each):

| API shape | Include surface (then route to security-test-matrix) |
|---|---|
| String input persisted to DB | SQL Injection |
| Returns user-supplied data in response | XSS |
| Path param `{id}` on user-owned resource | BOLA/IDOR |
| Admin-only / role-separated endpoint | BFLA |
| POST/PUT object body with extra fields possible | Mass Assignment |
| Every authenticated endpoint | Auth bypass (401 3-partition → `error-handling-strategy.md`) |
| Public / login / OTP endpoint | Rate limiting |
| Scheduler-triggered (no user context) | CRON_SECRET auth check |

Layer 2 tool-required surfaces (DAST, pentest) → log as risk per § Risk Classification; do not design TCs here.

### 2.5 Cross-Cutting Error TC Scope (Design Gate)

Before generating per-endpoint TCs, identify cross-cutting error scenarios that are shared across multiple endpoints and reserve **one shared TC set per spec file** — not per endpoint at L3. (Scope/count fixed here at design time; TC bodies are written once in the Detail phase and referenced by ID.)

Canonical definitions, TC IDs, and scenarios → **`error-handling-strategy.md § Cross-Cutting Error TCs`** and **§ Auth Error TCs**.

| Cross-cutting scenario | Applies to | Generate at |
|---|---|---|
| Malformed JSON | All POST / PUT / PATCH endpoints | Design phase — 1 shared TC |
| DB failure | All endpoints touching DB | Design phase — 1 TC per operation type (READ/WRITE/DELETE) |
| Auth 401 (missing / expired / malformed token) | All authenticated endpoints in group | Design phase — 3 shared TCs per auth group |
| Error envelope shape (`code` + `message`) | Every error TC | Assert rule applied at TC write time |

**Rule:** These TCs are referenced (not duplicated) by per-endpoint TCs via traceability. Do not regenerate at §3.4.

---

## Layer 3 — TC Count + Coverage Minimums

### 3.1 Per-Endpoint Minimum

Every endpoint in `scope_final` must have:
- ≥1 Happy Path TC (2xx)
- ≥1 Error TC (4xx)
- Auth TCs: 401 if auth required (shared across endpoints of same auth group)

**Response validation depth** (applies to every Happy Path TC): assert per `istqb-techniques.md § Assertion Depth per TC` + `security-test-matrix.md § Layer 1` (all applicable rows, incl. PII in Response).

### 3.2 POST / PATCH / PUT Tier Coverage

| Tier | TC type | Count |
|---|---|---|
| Tier 1 Happy | Distinct happy paths | 1 per distinct happy path |
| Tier 2 Invalid | Per-field, per-constraint | 1 per constraint (BVA/EP expanded) |
| Tier 3 Edge | Boundary, race, idempotency, state | 1+ per edge case identified in L1 |

### 3.3 DELETE Coverage

- Tier 1: Successful delete → 200/204 + verify resource 404 on follow-up GET
- Tier 2: Resource not found → 404; IDOR (wrong-user token) → 403
- Tier 3: Cascade effect on dependent resources → 1 TC asserting dependent state

### 3.4 Cross-Cutting TCs (L3 count)

Cross-cutting TCs are designed in Phase 2 (§2.5). At L3, apply counts from `error-handling-strategy.md § Cross-Cutting Error TCs` + `§ Auth Error TCs` directly — do not restate the counts here. Reference by TC ID — do not regenerate per endpoint.

### 3.5 Integration TC Minimums (§2)

Per integration point identified in Feature-Technical `## Integration Points`:
- ≥1 Happy path (external dep returns expected response)
- ≥1 Failure path (external dep times out / returns error)
- ≥1 Retry / fallback path (if documented)

**Async flow routing:**

| Async pattern | Postman-testable? | Action |
|---|---|---|
| **Polling** (client GETs `/status/{id}` until terminal state) | ✅ Yes | Use `setNextRequest` loop in Postman Runner: POST trigger → GET status (repeat until `status ≠ pending`, max N iterations) → assert terminal state |
| **Webhook / callback** (server POSTs to client URL) | ❌ No | Log `SKIP-ASYNC-WEBHOOK: {endpoint}` to `## Risk & Coverage Notes`; TC stub only (pre-condition: `webhook.site` relay — manual verify) |
| **Message queue consumer** (Kafka, SQS, RabbitMQ) | ❌ No | Log `SKIP-QUEUE: {queue name}` to `## Risk & Coverage Notes`; no TC generated |

**Failure-path pre-condition (Postman Mock Server):**
- When external dep has no test environment → configure Postman Mock Server to return scripted error (timeout / 503 / malformed body)
- TC pre-condition: `External dep: {name} mocked via Postman Mock Server returning {scenario}`
- If no mock available → log `GAP-MOCK: {endpoint}` to `## Risk & Coverage Notes` and skip failure-path TC

### 3.6 Backward-Compat TC Minimums (§3)

Per-change-type TC counts are defined inline in the **§1.5 Breaking-Change Detection** table (`Min TCs + assertion` column) — apply those directly. Breaking changes are BLOCKING (mandatory TCs).

**Version negotiation (if versioned API) — not in §1.5:**
- ≥1 TC per version header variant (`Accept: application/vnd.api+json;version=1` vs `version=2`)

### 3.7 NFR TC Minimums (§4)

**Performance:**
- ≥3 TCs per SLA threshold using BVA: below-threshold (pass), at-threshold (boundary), above-threshold (fail)
- If no SLA documented → skip Perf TCs, log to `## Risk & Coverage Notes`

**Security — Layer 1 (Postman-scriptable):**
- ≥1 TC per attack surface confirmed IN scope at §1.6 (surfaces routed via §2.4)
- Autoable = YES (TC assertions complete, Postman-only, no external tool required)
- Surface → technique routing → `security-test-matrix.md § Layer 3`

**Security DAST (Layer 2 — tool-required):**
- Out of Postman scope. Logged as `SKIP-DAST` at §1.6 NFR Scope Gate.
- No TCs generated here. Route to `/tas-security` or OWASP ZAP pipeline.

**Pentest recon:**
- Stub TCs only (Autoable = NO — human-led, Postman as tool-assist)
- Scope: documented recon scenarios only (not full engagement)

---

## API-Specific Dedup Extensions

Extends `tc-generation-strategy.md § 6 Deduplication Rules` for API tests. Auth/cross-cutting dedup → `error-handling-strategy.md § Auth Error TCs` + `§ Cross-Cutting Error TCs` (designed at §2.5, not per endpoint).

| Scenario | Dedup Rule | Dedup Key |
|---|---|---|
| Same field + same BVA constraint appears in multiple endpoints | 1 shared BVA TC set (4 TCs); reference by TC ID across endpoints | field name + constraint range |
| Same enum field (same values) validated by multiple endpoints | 1 shared EP TC set; reference across endpoints | field name + enum values |
| Same resource type accessed by multiple endpoints (GET + DELETE + PATCH) | 1 shared 404-MISSING + 404-DELETED TC; reference across | resource type + ownership model |
| IDOR check for same resource type | 1 TC per resource type per ownership model | resource type |
| Same state machine (status field) applies to multiple endpoints | 1 TC set per valid/invalid transition; reference across endpoints that trigger transitions | entity type + state machine |
| Same pagination params (`page`/`limit`/`cursor`) across endpoints | 1 shared pagination TC set (first/last/empty/beyond); reference across | param name + pagination style |

---

## Test Data Discipline

- All field values must be **concrete** in TC (e.g. `age: 35`, not `age: {value}` or `age: test-data`)
- Source hierarchy: TestChecklist §3 TD-NNN refs → OpenAPI example values → fabricated (flag in `## Risk & Coverage Notes`)
- If TD-NNN ref exists but value is abstract → BLOCK, ask user to specify concrete value
- For integration/NFR TCs: document environment/tool setup in Pre-Condition, not inline

---

## Risk Classification → Risk & Coverage Notes

**Purpose:** Define what `/tas-apitest-plan` must log to `## Risk & Coverage Notes` when a scenario cannot be covered by API tooling (Postman). An unrecorded known gap is not acceptable.

### Risk Trigger Checklist

When any of the following is detected during Steps 3–7, log to Risk section immediately — do NOT silently drop:

| Trigger | Risk level | Log entry format |
|---|---|---|
| Async webhook/callback endpoint — Postman cannot receive inbound HTTP | HIGH | `SKIP-ASYNC-WEBHOOK: {endpoint} — requires webhook relay (ngrok/webhook.site). Manual verify needed.` |
| Message queue consumer — no HTTP surface to test via Postman | HIGH | `SKIP-QUEUE: {endpoint/queue name} — queue protocol not testable via Postman. Needs dedicated queue test tool.` |
| Heavy load / stress / soak test — requires perf infra (k6/JMeter cluster) | MEDIUM | `SKIP-LOAD: {endpoint} — sustained load beyond Postman runner capacity. Defer to perf pipeline.` |
| Endpoint exists in code but excluded from spec (orphan, chose [A] in §1.2) | HIGH | `SKIP-ORPHAN: {method path} — implemented but out-of-scope. Untested attack surface. Justification: {reason}.` |
| Spec missing response schema / error codes / business rule → proceeding without | MEDIUM | `GAP-SPEC: {endpoint}.{field} — response schema/error code not documented. TC generated with assumed value; verify with BE.` |
| External dependency required for integration TC but no mock available | MEDIUM | `GAP-MOCK: {endpoint} — external dep {name} has no Postman Mock Server configured. Failure-path TC skipped.` |
| SLA not documented → Perf TCs skipped | LOW | `SKIP-PERF: No SLA documented for {endpoint group}. Perf TCs omitted per §3.7 rule.` |
| `[TECH-COVERAGE]` risk inherited from TestChecklist §5 | (inherit level) | `INHERITED: {risk text from TestChecklist §5 verbatim}` |

### Expert Review Escalation (Design-Time, Cost-Bounded)

**Purpose:** Some risks are *design defects* in Feature-Technical itself (insecure/inefficient/inconsistent design), not just coverage gaps — ISTQB Ch 5 Static Testing territory, requiring domain judgment `/tas-apitest-plan` should not self-judge. Escalate to the existing specialist reviewer agents (`security-reviewer`, `architect`, `database-reviewer`) instead of building new judgment logic here.

**Cost discipline (mandatory — do not skip):**
1. **Detect first, escalate never per-trigger.** Run the 3 deterministic triggers below (cheap string/table matching over digests already read in Phase 1.2/1.3 — zero extra reads) across the whole Feature-Technical. Collect all matches into a candidate list.
2. **Zero matches → zero agent calls.** Most runs should escalate nothing.
3. **One batched parallel dispatch** (end of Phase 1.5, not per-trigger) — same pattern as `/tas-security` Step 2. All matched candidates fire together, one Agent call each, run concurrently.
4. **Narrow scope per call** — pass ONLY the matched excerpt (the specific API Spec row / ERD Delta rows / Modules>Security line), never the full Feature-Technical file. This is what keeps token cost bounded regardless of Feature size.
5. **Cap:** if candidate list exceeds 5 items, escalate only the top 5 by risk (PII/auth > data-loss > perf) and log the rest as `GAP-REVIEW-DEFERRED` (Owner=PE) instead of spawning more agents.

| Trigger (deterministic, from digests already in memory) | Escalate to | Scope passed (excerpt only) |
|---|---|---|
| Endpoint touches PII/auth (per `security-test-matrix.md` surface classification) AND `## Modules > Security` has no matching `SEC-NN` | `security-reviewer` | That API Spec row + the `Security:` line of Modules |
| Endpoint is bulk/unpaginated on an entity with no row-count bound AND no NFR in `nfr_catalog` covers it | `architect` | That API Spec row + relevant `Data Flow` step |
| ≥2 endpoints write the same entity (per ERD Delta) with no version/ETag/idempotency-key field | `database-reviewer` | ERD Delta entity block + the 2 API Spec rows |

**Agent call template (per matched item):**
```
{Security|Performance|Data-consistency} design review — [excerpt only, not full doc].
Is this a real design defect, or acceptable as-is? Answer exactly one of:
CONFIRMED-DEFECT | NOT-AN-ISSUE | NEEDS-TECH-LEAD-DECISION
One-line reason.
```

**Verdict → Risk Registry:**
| Verdict | Action |
|---|---|
| `CONFIRMED-DEFECT` | Write §5 risk row now — RC per trigger (RC-SEC/RC-PERF/RC-DATA), `Reviewer: {agent}`, Level HIGH |
| `NOT-AN-ISSUE` | No risk row; do not log (avoids Risk Registry noise) |
| `NEEDS-TECH-LEAD-DECISION` | Write §5 risk row, Level MEDIUM, Status = ⏳ Active, Action = "Tech-lead review required" — follows the same Attempt-Answer BLOCK/ASK path as any unresolved gap, no new mechanism |

### Risk Log Format (per entry in output)

```
| Risk ID | Type | Endpoint / Scope | Description | Resolution |
|---|---|---|---|---|
| RISK-001 | SKIP-ASYNC-WEBHOOK | POST /webhooks/payment | Postman cannot receive inbound callback | Manual verify post-deploy; flag for automation if relay available |
| RISK-002 | GAP-SPEC | GET /orders/{id} | Response schema missing `status` enum values | Assumed [pending, confirmed, cancelled] — verify with BE |
```

**Rule:** Empty Risk table is acceptable only if all endpoints are fully covered. An empty table with known gaps = blocking review finding.

---

## Postman Output Mapping

| Markdown section | Postman folder | pm.test() pattern |
|---|---|---|
| §1 Contract | `Contract Tests` | `pm.response.to.have.status(N)` + schema assertion |
| §2 Integration | `Integration Tests` | `pm.response.to.have.status(N)` + dependency chain note |
| §3 Backward-Compat | `Backward-Compat Tests` | `pm.response.to.have.status(N)` + field existence check |
| §4 NFR — Perf | `NFR - Performance` | `pm.expect(pm.response.responseTime).to.be.below(N)` |
| §4 NFR — Security | `NFR - Security` | `pm.response.to.have.status(N)` + body not contain injection |
| §4 NFR — Pentest | `NFR - Pentest` | Skeleton only — `pm.test("Manual: {scenario}", () => {})` |

**NFR CI note:** Postman runner is for manual exploration only. For CI execution: Perf → k6, Security → OWASP ZAP. Document this in the collection description.