# Unnecessary Causes of Delays Between Agent Actions

Analysis of `packages/orchestrator/src/` (95 files) — identified delay sources ranked by impact.

---

## HIGH IMPACT

### 1. `streaming-executor.ts:171-192` — `waitAll()` busy-poll loop

```typescript
async waitAll(): Promise<void> {
  while (true) {
    const pending = Array.from(this.tools.values()).filter(
      e => e.state === "queued" || e.state === "executing"
    );
    if (pending.length === 0) break;
    const executing = pending.filter(e => e.promise);
    if (executing.length > 0) {
      await Promise.allSettled(executing.map(e => e.promise!));
      this.processQueue();
    } else {
      this.processQueue();
      await new Promise(r => setTimeout(r, 1));  // ← 1ms spin loop
    }
  }
}
```

**Problem:** When tools are queued but none have started (no `promise`), the loop spins with 1ms `setTimeout` between `processQueue()` calls. Each iteration costs at least 1ms of wall-clock delay. With many queued-but-blocked tools, this adds up.

**Fix:** Replace with Promise-based notification (event emitter or `AbortController`-based wait) so the loop yields until a tool completes or a new tool is enqueued.

---

### 2. `ollama-pool.ts:850-869` — Instance readiness probe loop

```typescript
await new Promise((r) => setTimeout(r, 500));  // ← 500ms between probes
```

**Problem:** When spawning a new Ollama instance, the code polls `/api/version` every **500ms** with a 2-second abort timeout. This adds **500-2000ms** of delay per new model instance. If the pool is empty and a model isn't loaded, every first request pays this cost.

**Fix:** Use exponential backoff (100ms → 200ms → 400ms) or a readiness event emitter instead of fixed polling.

---

### 3. `cascadeBackend.ts:105-156` — Sequential fallback chain

```typescript
const result = await backend.chatCompletion(request);
// On failure → try newBackend.chatCompletion(request);
// On failure → try primaryBackend.chatCompletion({ timeoutMs: 10_000 });
```

**Problem:** The cascade backend tries multiple backends in sequence. Each failure triggers a retry on the next backend. If the first backend is slow to fail (not crash), the full timeout (10s+) is paid before the fallback activates.

**Fix:** Start all backends simultaneously with `Promise.race()` and cancel losers on first success.

---

## MEDIUM IMPACT

### 4. `ollama-pool.ts:1190` — Unbounded slot waiter queue

```typescript
await new Promise<void>((resolve) => this.slotWaiters.push(resolve));
```

**Problem:** When all GPU slots are occupied, new requests queue as promises in `slotWaiters`. They only resolve when a slot frees up — but there's no timeout or progress feedback. Under heavy load, this can stall indefinitely.

**Fix:** Add a configurable timeout (e.g., 30s) and reject with a clear "no GPU available" error.

---

### 5. `ollama-pool.ts:1336` — GPU detection on every acquire

```typescript
const rawGpus = await this.gpuDetector();
```

**Problem:** GPU detection runs on every placement decision. If `gpuDetector()` calls `nvidia-smi` or similar system commands, this adds **10-100ms** per request.

**Fix:** Cache GPU detection results with a TTL (e.g., 30s). GPU topology rarely changes at runtime.

---

### 6. `tool-batching.ts:247-253` — Forced serial execution of non-concurrent tools

```typescript
for (const call of batch.calls) {
  results.push(await executeFn(call));
}
```

**Problem:** Write/shell tools are forced to run serially. If the agent produces 3+ write tools in one turn, they queue sequentially. No parallelism for independent writes.

**Fix:** Add a write-conflict analyzer (e.g., file-level locking) to allow parallel writes to different files.

---

### 7. `steeringIntake.ts:97` — 15s steering timeout

```typescript
timeoutMs = 15_000,
```

**Problem:** Steering intake has a 15-second timeout. If the model is slow or the prompt is large, this adds a full 15s delay before the steering response arrives.

**Fix:** Reduce to 5-8s for steering (which is advisory, not critical) or make it non-blocking with a best-effort result.

---

### 8. `verifierRunner.ts:161` — 60s test timeout

```typescript
timeout: 60_000,
```

**Problem:** Test execution has a 60-second timeout. If tests hang or are slow, this blocks the entire agent loop.

**Fix:** Add a per-test timeout and a total timeout with early termination on repeated failures.

---

## LOW-MEDIUM IMPACT

### 9. `ollama-pool.ts:1236,1257` — VRAM estimation per model

```typescript
const vramNeededMB = await this.estimateModelVramMB(model);
```

**Problem:** VRAM estimation runs before every spawn/placement. If it involves model metadata lookups or API calls, this adds latency.

**Fix:** Cache VRAM estimates keyed by model name with a TTL.

---

### 10. `streaming-executor.ts:21-26` — `stableValueKey` deep serialization

```typescript
function stableValueKey(value: unknown): string {
  if (value === null || typeof value !== "object") return JSON.stringify(value);
  if (Array.isArray(value)) return `[${value.map(stableValueKey).join(",")}]`;
  const record = value as Record<string, unknown>;
  return `{${Object.keys(record).sort().map((key) => ...).join(",")}}`;
}
```

**Problem:** Deep serialization of tool arguments for deduplication. For large args (e.g., file contents), this adds O(n) serialization cost per tool call.

**Fix:** Add a size cap (e.g., 10KB) — if the value exceeds it, use a hash instead of deep serialization.

---

### 11. `streaming-executor.ts:263-291` — Duplicate detection overhead

```typescript
private findPriorEquivalent(entry: StreamingToolEntry): StreamingToolEntry | null
private cloneDuplicateResult(entry: StreamingToolEntry): boolean
private mirrorPriorEquivalent(entry: StreamingToolEntry): boolean
```

**Problem:** Every tool call is checked against all prior calls for equivalence. This is O(n²) in the number of tools per turn.

**Fix:** Use a hash-based index (tool name + arg hash) for O(1) lookup instead of linear scan.

---

### 12. `ollama-pool.ts:1697` — Stale process cleanup timer

```typescript
const handle = setTimeout(async () => {
  const { cleanupStaleOllamaProcesses } = await import("./ollama-pool-cleanup.js");
  const report = await cleanupStaleOllamaProcesses({ ... });
}, ...);
```

**Problem:** Cleanup runs as a deferred `setTimeout` — adds latency before stale processes are actually cleaned up.

**Fix:** Run cleanup eagerly when a slot is freed, not on a timer.

---

### 13. `ollama-pool.ts:298,304` — `execSync` with 3s timeout

```typescript
{ encoding: "utf8", timeout: 3_000 },
```

**Problem:** Synchronous child processes block the event loop. If the command takes the full 3s, the entire process stalls.

**Fix:** Use `execFile` with `Promise` wrapper or `spawn` with timeout signal.

---

### 14. `preflightSnapshot.ts:284,311` — Multiple 1.5s probe timeouts

```typescript
timeout: 1500
```

**Problem:** Preflight checks run multiple probes, each with 1.5s timeout. If multiple probes fail, this adds 3-6s of startup delay.

**Fix:** Run probes in parallel with `Promise.allSettled()` and use the first success.

---

### 15. `tool-batching.ts:212-240` — Concurrency limit on reads

```typescript
export async function withConcurrencyLimit<T>(...) {
  // Uses a limiter to cap parallel reads
}
```

**Problem:** Concurrent-safe tools (reads) are limited by a concurrency cap. If the limit is too low, parallel reads are serialized unnecessarily.

**Fix:** Make the concurrency limit configurable and increase the default (e.g., from 2 to 8).

---

## Summary: Top 3 Fixes for Immediate Impact

| Priority | File | Change | Expected Savings |
|----------|------|--------|------------------|
| 1 | `streaming-executor.ts` | Replace 1ms spin loop with Promise notification | Eliminates busy-poll latency entirely |
| 2 | `ollama-pool.ts` | Cache GPU detection with 30s TTL | Eliminates 10-100ms per request |
| 3 | `cascadeBackend.ts` | Parallelize backend fallbacks with `Promise.race()` | Eliminates cascading timeout accumulation |