[
  {
    "id": "cost-001",
    "name": "API extraction",
    "question": "What authentication methods does the API support?",
    "context": "# Authentication API Reference\n\nThe Gateway API supports three authentication methods for securing client requests.\n\n## 1. API Key Authentication\n\nPass your API key via the `X-API-Key` header. Keys are generated in the Developer Portal under Settings > API Keys. Each key has configurable rate limits and scope restrictions. Keys can be rotated without downtime using the dual-key mechanism: generate a new key, update clients, then revoke the old key.\n\n## 2. OAuth 2.0 Bearer Tokens\n\nFor user-delegated access, the API supports OAuth 2.0 Authorization Code flow with PKCE. Request tokens from `POST /oauth/token` with grant_type=authorization_code. Access tokens expire after 3600 seconds. Refresh tokens are valid for 30 days and support automatic rotation.\n\n## 3. Mutual TLS (mTLS)\n\nFor service-to-service communication, mTLS provides certificate-based authentication. Upload your client certificate via the Admin Console. The API validates the certificate chain against the configured CA bundle. Certificate pinning is supported but optional.",
    "category": "extraction"
  },
  {
    "id": "cost-002",
    "name": "Architecture summary",
    "question": "Summarize the key architectural decisions and their trade-offs.",
    "context": "# System Architecture Document\n\n## Overview\n\nThe platform uses a microservices architecture deployed on Kubernetes, with an event-driven backbone for inter-service communication. This document captures the major architectural decisions made during the Q3 2025 redesign.\n\n## Decision 1: Event Sourcing for Order Management\n\nWe adopted event sourcing for the order management domain. Every state change is captured as an immutable event in an append-only log (Apache Kafka). The current state is derived by replaying events. Trade-off: increased storage requirements (~3x compared to state-based) and higher complexity for simple queries, but we gain complete audit trails, temporal queries, and the ability to rebuild read models.\n\n## Decision 2: CQRS for Read/Write Separation\n\nCommand and Query Responsibility Segregation separates the write model (event store) from read models (PostgreSQL materialized views). Writes go through the command bus, reads hit optimized projections. Trade-off: eventual consistency (typically <100ms lag) and operational complexity of maintaining projections, but read performance improved 15x and we can scale read/write independently.\n\n## Decision 3: Service Mesh with Istio\n\nAll inter-service communication flows through Istio sidecars. This provides mTLS between services, circuit breaking, retries with exponential backoff, and observability (distributed tracing via Jaeger). Trade-off: ~2ms latency overhead per hop and significant operational complexity, but unified security policy enforcement and deep traffic visibility.\n\n## Decision 4: GraphQL Federation\n\nThe API gateway uses Apollo Federation to compose a unified GraphQL schema from individual service schemas. Each team owns their subgraph. Trade-off: schema governance overhead and potential N+1 query issues, but teams can iterate independently and clients get a single endpoint with exactly the data they need.\n\n## Decision 5: Multi-Region Active-Active\n\nThe system operates in active-active mode across us-east-1 and eu-west-1. CockroachDB handles cross-region replication with serializable isolation. DNS-based routing directs users to the nearest region. Trade-off: significantly higher infrastructure cost (~2.3x) and complex conflict resolution for concurrent writes, but we achieve <50ms latency globally and survive full region failures.",
    "category": "summarization"
  },
  {
    "id": "cost-003",
    "name": "Migration reasoning",
    "question": "Given the constraints described, should the team proceed with the database migration to PostgreSQL 16 during the holiday freeze, or wait until Q1? Explain your reasoning step by step.",
    "context": "# Database Migration Planning: PostgreSQL 14 to 16\n\n## Current State\n- Running PostgreSQL 14.9 on AWS RDS Multi-AZ\n- 847 GB data across 23 databases\n- Peak load: 12,000 queries/second during business hours\n- End-of-life for PG 14 community support: November 2026\n\n## Migration Benefits (PG 16)\n- Logical replication improvements: parallel apply for large transactions\n- Performance: up to 30% improvement for bulk operations via SIMD JSON parsing\n- MERGE command support (SQL standard compliance)\n- pg_stat_io view for I/O monitoring\n- Security: libpq now supports Kerberos credential delegation\n\n## Holiday Freeze Context\n- Company holiday freeze: December 15 - January 5\n- All production changes require VP-level approval during freeze\n- On-call team reduced to 2 engineers (vs normal 6)\n- Customer traffic drops 40% during holidays (lower risk window)\n- Q1 has two major feature launches planned (weeks 3 and 7)\n\n## Risk Factors\n- Three extensions need compatibility verification: PostGIS 3.3, pg_partman 4.7, timescaledb 2.11\n- Application connection pooling (PgBouncer 1.19) needs config update for PG 16 protocol changes\n- ORM (Prisma 5.x) has known issue with PG 16 MERGE — workaround available but untested in production\n- Rollback requires full RDS snapshot restore (~45 minutes for 847 GB)\n- Two critical batch jobs run nightly at 02:00 UTC and cannot be interrupted\n\n## Team Assessment\n- DBA team confidence level: 7/10 (comfortable with process, concerned about extension compat)\n- Dev team readiness: 6/10 (integration tests pass, but limited PG 16 staging time)\n- Staging environment has been running PG 16 for 3 weeks with synthetic load\n- One P2 bug found in staging: connection timeout under high concurrent MERGE operations\n\n## Compliance Requirements\n- SOC 2 audit scheduled for February — auditors expect documented change management\n- GDPR data residency constraints require EU region migration to be completed within same maintenance window\n- PCI DSS requires change approval documentation with rollback evidence",
    "category": "reasoning"
  },
  {
    "id": "cost-004",
    "name": "Framework comparison",
    "question": "Compare the three deployment strategies described and recommend which one is best suited for a team with limited DevOps experience.",
    "context": "# Deployment Strategy Comparison\n\n## Strategy A: Blue-Green Deployment\n\nMaintain two identical production environments (blue and green). At any time, one serves live traffic while the other is idle. To deploy: push new code to idle environment, run smoke tests, then switch the load balancer. Rollback is instant — switch back to the previous environment.\n\nResource cost: 2x infrastructure at all times. Complexity: moderate — requires automated environment provisioning and LB switching. Downtime: zero during switchover. Database handling: challenging — requires backward-compatible schema migrations or database-per-environment.\n\nBest for: teams with budget for double infrastructure, need instant rollback, relatively stable database schemas.\n\n## Strategy B: Canary Deployment\n\nRoute a small percentage (1-5%) of traffic to the new version while 95-99% stays on the old version. Gradually increase the percentage while monitoring error rates, latency, and business metrics. If anomalies are detected, route all traffic back to the old version.\n\nResource cost: 1.05-1.1x infrastructure during rollout. Complexity: high — requires sophisticated traffic routing (service mesh or smart LB), real-time monitoring, automated rollback triggers. Downtime: zero. Database handling: both versions must work with same database, requiring backward-compatible migrations.\n\nBest for: high-traffic services where bugs in new versions could have significant blast radius, teams with strong observability.\n\n## Strategy C: Rolling Update\n\nReplace instances of the old version with the new version one at a time (or in small batches). Kubernetes does this natively with Deployment resources. At any point during rollout, both old and new versions handle traffic.\n\nResource cost: 1.1-1.25x infrastructure during rollout. Complexity: low — built into Kubernetes, minimal configuration (maxSurge, maxUnavailable). Downtime: zero if configured correctly. Database handling: same as canary — requires backward-compatible migrations.\n\nBest for: Kubernetes-native teams, services that handle mixed-version traffic well, teams wanting simplicity.",
    "category": "comparison"
  },
  {
    "id": "cost-005",
    "name": "Incident synthesis",
    "question": "Synthesize the incident reports below into a unified root cause analysis and propose systemic improvements.",
    "context": "# Incident Reports — Q3 2025\n\n## INC-2847: Payment Processing Outage (August 3, 14:22-15:47 UTC)\n\nSeverity: P1 | Duration: 85 minutes | Impact: 100% of payment transactions failed\n\nTimeline:\n- 14:22 — Monitoring alert: payment-service error rate >50%\n- 14:25 — On-call engineer begins investigation\n- 14:32 — Root cause identified: expired TLS certificate on payment gateway integration\n- 14:45 — Certificate renewal initiated via automated pipeline\n- 14:58 — New certificate deployed but service still failing\n- 15:12 — Discovery: PgBouncer connection pool exhausted due to retry storm from failed payments\n- 15:30 — PgBouncer restarted, connection pool drained\n- 15:47 — Full recovery confirmed\n\nRoot cause: TLS certificate auto-renewal cron job was disabled during June infrastructure migration and never re-enabled. Certificate expired after 90-day Let's Encrypt cycle.\n\nContributing factors: No monitoring on certificate expiry dates. Retry logic in payment-service uses fixed 1-second intervals (no exponential backoff), causing connection pool exhaustion.\n\n## INC-2901: Search Degradation (August 19, 09:15-10:30 UTC)\n\nSeverity: P2 | Duration: 75 minutes | Impact: Search latency increased from 200ms to 8s (p99)\n\nTimeline:\n- 09:15 — Latency alert triggered on search-service\n- 09:22 — Investigation reveals Elasticsearch cluster yellow status\n- 09:28 — One ES data node (es-data-04) unresponsive, triggering shard rebalancing\n- 09:45 — Node es-data-04 confirmed OOM-killed by Kubernetes (memory limit: 32GB, actual usage: 31.8GB)\n- 10:00 — Node restarted with increased memory limit (48GB)\n- 10:15 — Shard rebalancing complete\n- 10:30 — Latency returned to normal\n\nRoot cause: A bulk indexing job (product catalog refresh) was scheduled concurrently with peak search traffic. The indexing job consumed excessive heap memory on es-data-04 because it processed 50,000 documents per batch instead of the recommended 5,000.\n\nContributing factors: No resource isolation between indexing and search workloads. Bulk batch size was increased from 5,000 to 50,000 three months ago without load testing.\n\n## INC-2956: API Gateway Cascade Failure (September 2, 16:00-17:15 UTC)\n\nSeverity: P1 | Duration: 75 minutes | Impact: All API endpoints returned 503 for 60% of requests\n\nTimeline:\n- 16:00 — Marketing campaign launched, driving 3x normal traffic\n- 16:05 — User-service auto-scaling triggered (3 → 12 pods)\n- 16:08 — New user-service pods pulling container image (2.1GB) from registry\n- 16:12 — Container registry rate limit hit (DockerHub free tier: 100 pulls/6 hours)\n- 16:15 — 9 of 12 user-service pods stuck in ImagePullBackoff\n- 16:18 — Remaining 3 pods overwhelmed, latency spikes to 30s\n- 16:22 — Istio circuit breaker trips on user-service, returning 503\n- 16:25 — Cascade: order-service and cart-service depend on user-service, both start failing\n- 16:40 — Team switches to private ECR registry mirror\n- 16:55 — Images pulled from ECR, pods come up\n- 17:15 — Full recovery, circuit breakers reset\n\nRoot cause: Production Kubernetes cluster configured to pull container images from DockerHub free tier instead of private registry mirror. Combined with 2.1GB image size, auto-scaling events exhausted the rate limit.\n\nContributing factors: No pre-pulled images on nodes. Container image not optimized (2.1GB includes dev dependencies). Auto-scaling policy scales too aggressively (100% pod increase per step). No alerting on container registry rate limits.\n\n## INC-3012: Data Pipeline Delay (September 15, 03:00-11:00 UTC)\n\nSeverity: P2 | Duration: 8 hours | Impact: Analytics dashboards showed stale data (8+ hours old)\n\nTimeline:\n- 03:00 — Nightly ETL job starts, processing 2.3TB of event data\n- 03:45 — Spark executor OOM on largest partition (events from top-3 customers = 40% of data)\n- 04:00 — Spark retry policy kicks in (3 retries, same configuration)\n- 06:00 — All retries exhausted, job marked as failed\n- 08:30 — Morning shift notices stale dashboards\n- 09:00 — Engineer investigates, identifies skewed partition\n- 10:00 — Job rerun with increased executor memory (16GB → 32GB) and partition balancing\n- 11:00 — Backfill complete\n\nRoot cause: Data skew — three enterprise customers generated 40% of daily events, causing one Spark partition to exceed executor memory limits.\n\nContributing factors: No data skew monitoring. No alerting on ETL job failures (only dashboards stale alert exists). Retry policy retries with same config (insanity). No partition rebalancing strategy for skewed keys.",
    "category": "synthesis"
  },
  {
    "id": "cost-006",
    "name": "Config extraction",
    "question": "Extract all environment variables and their default values from this configuration reference.",
    "context": "# Service Configuration Reference\n\nThe worker service reads configuration from environment variables at startup. All variables are optional and have sensible defaults.\n\n## Server Settings\n\n`PORT` — HTTP server listen port. Default: `8080`. Must be between 1024 and 65535.\n\n`HOST` — Bind address. Default: `0.0.0.0`. Set to `127.0.0.1` for local-only access.\n\n`WORKER_CONCURRENCY` — Number of parallel worker threads. Default: `4`. Recommended: set to CPU core count.\n\n## Database\n\n`DATABASE_URL` — PostgreSQL connection string. Default: `postgresql://localhost:5432/worker_db`. Supports connection pooling parameters via query string.\n\n`DB_POOL_SIZE` — Maximum database connections. Default: `10`. Should not exceed `max_connections / number_of_instances`.\n\n`DB_TIMEOUT_MS` — Query timeout in milliseconds. Default: `30000`.\n\n## Cache\n\n`REDIS_URL` — Redis connection string. Default: `redis://localhost:6379/0`.\n\n`CACHE_TTL_SECONDS` — Default cache entry TTL. Default: `300`.\n\n`CACHE_PREFIX` — Key prefix for cache entries. Default: `worker:`.",
    "category": "extraction"
  }
]