# Task 2023 — sidebar-sessions async reads Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Stop `computeSessionList` from blocking the one event loop for the whole duration of its filesystem walk, without changing what the endpoint returns. **Architecture:** The route already sits behind the Task 1986 per-account cache and its compute is already `async` and awaited. The block is not the awaiting; it is that every read inside the compute is synchronous, so the loop cannot run anything else from the first `readFileSync` to the last. Replacing the per-row reads with `node:fs/promises` gives the loop a yield point per row. Directory enumeration is left alone on measurement, not preference (see Global Constraints). **Tech Stack:** TypeScript, Node 22, Hono, vitest. ## Global Constraints - **Measured cost split, maxy 2026-07-26, 52 files / 133.9 MB / 38,042 lines, two identical runs:** `enumMs=1`, `readMs=284.5` then `277.4`, `parseMs=212.3` then `216.6`. - **The four `readdirSync` calls cost 1 ms and are NOT converted.** `enumerateJsonls` is imported by six other server modules; forking it for 1 ms would trade a measurable regression risk for no measurable gain. This is a deliberate deviation from the task file's problem statement, which names lines 216, 227, 240 and 255 alongside the reads that do cost. - **JSON parsing is ~215 ms of pure CPU and no async change removes it.** The deliverable is that it stops arriving as one 215 ms block and arrives as ~52 chunks separated by I/O yields. Do not claim the parse cost is eliminated. - **Reads stay sequential.** The current loop holds one `body` at a time and lets it go out of scope each iteration. `Promise.all` over 52 files would hold 133.9 MB at once. Sequential `await` yields the loop without changing the memory profile. - **No behaviour change.** Response shape, field values, ordering, census log lines and cache states are all byte-identical. - **Existing sync exports keep their signatures.** `enumerateJsonls`, `loadUserTitles`, `readSidecarMeta` and `readBridgeIds` are consumed by `server/index.ts`, `public-reader.ts`, `webchat.ts`, `whatsapp-reader.ts`, `files.ts` and two test files. None of those are in scope. - **One parser per format.** Where a sync and an async reader both exist, they share a single exported pure parse function. Two copies of a parse body is a drift registry and is not acceptable. --- ## File Structure - `server/routes/admin/sidebar-sessions.ts` — the only production file modified. Gains `parseUserTitles`, `parseSidecarMeta`, `loadUserTitlesAsync`, `readSidecarMetaAsync`; `computeSessionList`'s row loop switches to `fs/promises`. - `server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts` — new, holds this task's own tests. --- ### Task 1: Split the two parsers out of their sync readers **Files:** - Modify: `platform/ui/server/routes/admin/sidebar-sessions.ts:308-335` (`loadUserTitles`), `:371-402` (`readSidecarMeta`) - Test: `platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts` **Interfaces:** - Produces: `parseUserTitles(raw: string): Map` and `parseSidecarMeta(raw: string): SidecarMeta`. Both take the file's raw text and never touch the filesystem. Both degrade to empty/null on a parse failure or wrong shape, exactly as the current readers do. - Consumes: nothing. - [ ] **Step 1: Write the failing test** ```ts import { describe, it, expect } from 'vitest' import { parseUserTitles, parseSidecarMeta } from '../sidebar-sessions' describe('parseUserTitles', () => { it('keeps a trimmed string title and drops every malformed entry', () => { const raw = JSON.stringify({ good: { title: ' renamed ' }, blankTitle: { title: ' ' }, wrongType: { title: 42 }, notAnObject: 'nope', '': { title: 'no id' }, }) const out = parseUserTitles(raw) expect(out.get('good')).toBe('renamed') expect([...out.keys()]).toEqual(['good']) }) it('returns an empty map for unparseable or non-object JSON', () => { expect(parseUserTitles('{not json').size).toBe(0) expect(parseUserTitles('[1,2,3]').size).toBe(0) expect(parseUserTitles('null').size).toBe(0) }) }) describe('parseSidecarMeta', () => { it('reads every field and filters non-string bridgeIds', () => { const raw = JSON.stringify({ bridgeIds: ['a', '', 7, 'b'], role: 'admin', channel: 'webchat', senderId: '+447700900000', startedAt: '2026-07-26T00:00:00.000Z', personId: 'p1', adminUserId: 'u1', visitorId: 'v1', agentSlug: 'slug', accountId: 'acct', }) expect(parseSidecarMeta(raw)).toEqual({ bridgeIds: ['a', 'b'], role: 'admin', channel: 'webchat', senderId: '+447700900000', startedAt: '2026-07-26T00:00:00.000Z', personId: 'p1', adminUserId: 'u1', visitorId: 'v1', agentSlug: 'slug', accountId: 'acct', }) }) it('returns the empty shape for unparseable JSON', () => { expect(parseSidecarMeta('{not json')).toEqual({ bridgeIds: [], role: null, channel: null, senderId: null, startedAt: null, personId: null, adminUserId: null, visitorId: null, agentSlug: null, accountId: null, }) }) }) ``` - [ ] **Step 2: Run test to verify it fails** Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts` Expected: FAIL — `parseUserTitles is not a function` (the export does not exist yet). - [ ] **Step 3: Write minimal implementation** Replace the body of `loadUserTitles` from the `JSON.parse` onward with a call to a new exported pure function, and do the same for `readSidecarMeta`. The sync readers keep their exact signatures and their `readFileSync` + catch. ```ts /** Pure half of {@link loadUserTitles}: parse the on-disk `session-titles.json` * text. Shared with the async reader so one format has one parser. */ export function parseUserTitles(raw: string): Map { const out = new Map() let parsed: unknown try { parsed = JSON.parse(raw) } catch { return out } if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return out for (const [sessionId, value] of Object.entries(parsed as Record)) { if (!sessionId) continue if (!value || typeof value !== 'object') continue const v = value as { title?: unknown } if (typeof v.title !== 'string') continue const trimmed = v.title.trim() if (!trimmed) continue out.set(sessionId, trimmed) } return out } export function loadUserTitles(accountDir: string | null): Map { if (!accountDir) return new Map() let raw: string try { raw = readFileSync(join(accountDir, 'session-titles.json'), 'utf8') } catch { return new Map() } return parseUserTitles(raw) } ``` ```ts // A factory, not a shared constant. `{ ...CONST }` would copy the REFERENCE to // one `bridgeIds` array into every empty result, so a single caller mutating it // would corrupt every future empty sidecar in the process. function emptySidecarMeta(): SidecarMeta { return { bridgeIds: [], role: null, channel: null, senderId: null, startedAt: null, personId: null, adminUserId: null, visitorId: null, agentSlug: null, accountId: null, } } /** Pure half of {@link readSidecarMeta}: parse a `.meta.json` sidecar's text. * Shared with the async reader so one format has one parser. */ export function parseSidecarMeta(raw: string): SidecarMeta { let parsed: unknown try { parsed = JSON.parse(raw) } catch { return emptySidecarMeta() } if (!parsed || typeof parsed !== 'object') return emptySidecarMeta() const r = parsed as { bridgeIds?: unknown; role?: unknown; channel?: unknown; senderId?: unknown; startedAt?: unknown; personId?: unknown; adminUserId?: unknown; visitorId?: unknown; agentSlug?: unknown; accountId?: unknown } const bridgeIds = Array.isArray(r.bridgeIds) ? r.bridgeIds.filter((b): b is string => typeof b === 'string' && b.length > 0) : [] return { bridgeIds, role: typeof r.role === 'string' ? r.role : null, channel: typeof r.channel === 'string' ? r.channel : null, senderId: typeof r.senderId === 'string' ? r.senderId : null, startedAt: typeof r.startedAt === 'string' ? r.startedAt : null, personId: typeof r.personId === 'string' ? r.personId : null, adminUserId: typeof r.adminUserId === 'string' ? r.adminUserId : null, visitorId: typeof r.visitorId === 'string' ? r.visitorId : null, agentSlug: typeof r.agentSlug === 'string' ? r.agentSlug : null, accountId: typeof r.accountId === 'string' ? r.accountId : null, } } export function readSidecarMeta(metaPath: string): SidecarMeta { let raw: string try { raw = readFileSync(metaPath, 'utf8') } catch { return emptySidecarMeta() } return parseSidecarMeta(raw) } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts server/routes/admin/__tests__/sidebar-sessions.test.ts server/routes/admin/__tests__/resolve-title.test.ts` Expected: PASS on all three files. The two pre-existing files are the regression guard on the sync readers whose bodies just moved. - [ ] **Step 5: Commit** ```bash git add maxy-code/platform/ui/server/routes/admin/sidebar-sessions.ts maxy-code/platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts git commit -m "refactor(2023): split the sidecar and user-title parsers out of their sync readers" ``` --- ### Task 2: Add the async readers and switch the row loop onto them **Files:** - Modify: `platform/ui/server/routes/admin/sidebar-sessions.ts:70` (imports), `:572` (`loadUserTitles` call site), `:642-655` (row loop reads) - Test: `platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts` **Interfaces:** - Consumes: `parseUserTitles`, `parseSidecarMeta`, `emptySidecarMeta` from Task 1. - Produces: `loadUserTitlesAsync(accountDir: string | null): Promise>` and `readSidecarMetaAsync(metaPath: string): Promise`. Same degradation rules as their sync twins. - [ ] **Step 1: Write the failing test** The load-bearing assertion is that the loop yields. A test that only checks the returned rows would pass just as well against the synchronous version, which is the dead-signal case of Task 1730. So the test races a timer against the compute: with synchronous reads the timer cannot fire until the walk finishes; with async reads it fires during it. ```ts import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { loadUserTitlesAsync, readSidecarMetaAsync } from '../sidebar-sessions' let dir: string beforeEach(() => { dir = mkdtempSync(join(tmpdir(), 'sbs-async-')) }) afterEach(() => { rmSync(dir, { recursive: true, force: true }) }) describe('loadUserTitlesAsync', () => { it('reads and parses the same file as the sync twin', async () => { writeFileSync(join(dir, 'session-titles.json'), JSON.stringify({ s1: { title: 'renamed' } })) expect((await loadUserTitlesAsync(dir)).get('s1')).toBe('renamed') }) it('returns an empty map for a null accountDir and for a missing file', async () => { expect((await loadUserTitlesAsync(null)).size).toBe(0) expect((await loadUserTitlesAsync(join(dir, 'absent'))).size).toBe(0) }) }) describe('readSidecarMetaAsync', () => { it('reads and parses the same file as the sync twin', async () => { const p = join(dir, 'x.meta.json') writeFileSync(p, JSON.stringify({ role: 'admin', channel: 'webchat' })) const meta = await readSidecarMetaAsync(p) expect(meta.role).toBe('admin') expect(meta.channel).toBe('webchat') }) it('returns the empty shape for a missing file', async () => { expect((await readSidecarMetaAsync(join(dir, 'absent.meta.json'))).bridgeIds).toEqual([]) }) }) describe('the row loop yields to the event loop', () => { it('lets a pending timer fire while the walk is still running', async () => { const root = join(dir, 'projects', 'slug') mkdirSync(root, { recursive: true }) // Bodies large enough that the walk cannot complete inside one tick. const line = JSON.stringify({ type: 'user', message: { role: 'user', content: 'x'.repeat(400) } }) for (let i = 0; i < 40; i++) { const id = `0000000${i.toString().padStart(2, '0')}-0000-4000-8000-000000000000` writeFileSync(join(root, `${id}.jsonl`), `${line}\n`.repeat(500)) } let firedDuringWalk = false const walk = readAllBodiesForTest(join(dir, 'projects')) setTimeout(() => { firedDuringWalk = true }, 0) await walk expect(firedDuringWalk).toBe(true) }) }) ``` - [ ] **Step 2: Run test to verify it fails** Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts` Expected: FAIL — `loadUserTitlesAsync is not a function`, and `readAllBodiesForTest is not defined`. - [ ] **Step 3: Write minimal implementation** Add the promises import beside the existing sync one, add the two async readers, and export the loop's read half as `readAllBodiesForTest` so the yield test drives the shipped code rather than a hand-copied twin. ```ts import { readdirSync, readFileSync, statSync } from 'node:fs' import { readFile, stat } from 'node:fs/promises' ``` ```ts /** Async twin of {@link loadUserTitles}. Same parser, same degradation. Used by * computeSessionList so the request path never blocks the loop on this read. */ export async function loadUserTitlesAsync(accountDir: string | null): Promise> { if (!accountDir) return new Map() let raw: string try { raw = await readFile(join(accountDir, 'session-titles.json'), 'utf8') } catch { return new Map() } return parseUserTitles(raw) } /** Async twin of {@link readSidecarMeta}. Same parser, same degradation. */ export async function readSidecarMetaAsync(metaPath: string): Promise { let raw: string try { raw = await readFile(metaPath, 'utf8') } catch { return emptySidecarMeta() } return parseSidecarMeta(raw) } /** The row loop's read half, exported only so the yield test drives the shipped * code. Sequential by design: `Promise.all` here would hold every body in * memory at once (133.9 MB measured on maxy 2026-07-26). */ export async function readAllBodiesForTest(projectsRoot: string): Promise { let n = 0 for (const { path } of enumerateJsonls(projectsRoot)) { try { await readFile(path, 'utf8'); await stat(path); n++ } catch { continue } } return n } ``` In `computeSessionList`, change line 572 and the row loop's reads: ```ts const userTitles = await loadUserTitlesAsync(accountDir) ``` ```ts let body: string let mtimeMs: number try { body = await readFile(path, 'utf8') mtimeMs = (await stat(path)).mtimeMs } catch { continue } ``` ```ts const { bridgeIds, role, channel: sidecarChannel, senderId, personId, adminUserId, accountId: rowAccountId } = await readSidecarMetaAsync(metaPath) ``` - [ ] **Step 4: Run tests to verify they pass** Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts` Expected: PASS, including the yield test. - [ ] **Step 5: Mutation-check the yield test** Temporarily revert `readAllBodiesForTest` to `readFileSync` / `statSync` and re-run. Expected: the yield test FAILS with `firedDuringWalk` false. Restore the async version. A yield test that passes against synchronous reads is measuring nothing. - [ ] **Step 6: Typecheck and run the full change surface** Run: `cd maxy-code/platform/ui && npx tsc --noEmit -p tsconfig.json` Expected: 0 errors. Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/ server/lib/__tests__/route-cache.test.ts server/__tests__/edge-request-timing.test.ts` Expected: PASS. `route-cache-watchers.test.ts` fails only when batched with its siblings and passes 4/4 alone; that is Task 2014 and predates this change. - [ ] **Step 7: Commit** ```bash git add maxy-code/platform/ui/server/routes/admin/sidebar-sessions.ts maxy-code/platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts git commit -m "perf(2023): read session bodies and sidecars asynchronously so the walk yields the loop" ``` --- ### Task 3: Correct the task file's own criterion and figures **Files:** - Modify: `maxy-code/.tasks/2023-sidebar-sessions-sync-fs-blocks-event-loop.md` **Interfaces:** none. - [ ] **Step 1: Replace the p99 verification criterion** The task says a p99 approaching the request durations confirms and a low p99 refutes. Fourteen consecutive windows measured on maxy read `p99=51` or `52` including one with `max=663`, because one blocked tick out of ~590 samples sits nowhere near the 99th percentile. Rewrite the Observability and Verification sections to turn on `max`, which is the statistic that sees a single sub-second block and which Task 2024 also shipped. - [ ] **Step 2: Replace the 620 ms figure** The `p50 620 ms across 27,280 requests` sample is undatable and mostly predates the Task 1986 route cache, which landed 2026-07-25. State the datable post-cache distribution instead: `n=38 p50=2 p90=603 p99=678 max=1161`, and that the cost is one full walk per 30 s poll because the poll interval equals `BACKSTOP_MS`. - [ ] **Step 3: Record the enumeration deviation** State that the four `readdirSync` calls were measured at 1 ms and deliberately left synchronous, with the six other importers of `enumerateJsonls` as the reason. - [ ] **Step 4: Commit** ```bash git add maxy-code/.tasks/2023-sidebar-sessions-sync-fs-blocks-event-loop.md git commit -m "docs(2023): correct the p99 criterion and the pre-cache 620ms figure" ``` --- ## Verification (Phase 4, on-device) The unit tests prove the loop yields. They cannot prove the lag falls on real data, so the sprint's claim rests on a device reading taken the same way as the before reading: - Before, already recorded: fourteen windows, `max` 310 to 708 ms in every window containing a `cache=stale-backstop` recompute, `max` 52 to 53 ms in every window without one. - After: the same grep across a window of real dashboard traffic. A window containing a `stale-backstop` recompute should no longer show a `max` near the recompute's own `ms=`. - Refutation case: if `max` still tracks the recompute duration after the change, the remaining block is the ~215 ms of parsing and the change did not achieve its goal. That is a reportable result, not a failed change. This step needs an installer publish to reach the device and is therefore operator-gated.