# The spawn-log write lock

`agents/<paName>/spawn-log.json` is the record of every dispatch. Every change to it is a read-modify-write under a lock beside the file, `spawn-log.json.lock`, then a temp file renamed over the spawn-log. The code is `packages/core/src/spawn-log/write.mjs` (`withSpawnLogLock`, `updateSpawnLog`, `writeSpawnLogAtomic`, `readSpawnLogLock`, `removeSpawnLogLock`). This page is the lock's design: what it guarantees, what each state and crash point does on each class of filesystem, and the test that proves each cell.

The design fails safe. A lock is broken automatically only where its holder's death is provable. Every other held lock blocks: after the 10 s timeout the writer fails with a message that names the holder and the one command that clears it, `pa dispatch unlock`. The trade is deliberate: a rare visible block, cleared by one named command, instead of a rare silent loss of a record.

## The supported envelope

The lock is supported on one host with its estate on a local filesystem: every process that writes the spawn-log runs on that one machine, and the spawn-log's folder is on a disk that machine mounts locally (APFS, ext4, NTFS, or exFAT and other local volumes without hard links). Every guarantee, residual and table cell on this page holds inside that envelope, and none is claimed outside it.

Inside the envelope the design relies on what a local filesystem gives: an exclusive create (`wx`), `link()` and `rename()` are atomic, and a directory listing taken after a create has completed shows that file. It also relies on every writer of the folder being on this host, so the one automatic break's pid check is a check of the holder's own process.

A spawn-log folder shared across hosts (a network share, or one folder written from two machines at once) is unsupported, for two reasons:

- **Two hosts with one hostname.** The one automatic break checks the holder's pid on this host. A live holder on another host that reports the same hostname has a pid that does not exist here, so it reads as dead and its live lock is broken automatically. If that holder has already passed its fence, its commit can land over a newer writer's, and a record is lost.
- **A directory listing that omits a just-completed create.** Breakers of one dead lock run one at a time only because each lists the folder after creating its claim and sees the other's claim. On a share whose listing can omit a claim another host has just created, two breakers can both go on. The second can then move the fresh lock the first published for a live holder, and a commit inside that holder's fence-to-rename window can lose a record.

Some SMB and network shares have no hard links either. A network share is not a local filesystem, so it is outside the envelope even when only one host uses it; the second column of the table below describes the class without hard links on a local volume.

If a spawn-log ever has to live on a filesystem shared across hosts, the path is a lock-free write path (one record file per dispatch, claimed by exclusive create, with the spawn-log as an index built from those files), not this lock.

## Guarantees

Inside the envelope, on every path that does not use `pa dispatch unlock --force`:

1. **No record is ever lost.** A writer that reports success has its record in the spawn-log, and a later writer never commits a snapshot that drops it. Every commit is fenced: immediately before the commit rename, the writer re-reads the lock and checks that it still carries the writer's own token. If it does not, the writer does not commit; it runs its whole read-modify-write again under a fresh lock, within the lock timeout, and fails with a lock error (exit 2 from `pa dispatch`) when the timeout runs out. A failure is reported, never a silent drop.
2. **A live holder never loses its lock to an automatic step.** No writer ever breaks a lock whose holder might be alive: not on age, not past any window, not by a name another process can re-create. The only automatic break is a lock whose holder is on this host and whose pid is dead. Breakers of one dead lock run one at a time (each takes a claim, below), and a release only ever moves its own lock.

The residual windows: each starts with `pa dispatch unlock --force`, and each is named with its width and what would have to happen inside it. `--force` removes a lock whatever its holder, after printing it. That is the operator's decision, never an automatic step, and it is why every `--force` hint says to use it only once the holder's process is gone or stuck, never while it is still running.

- **A forced unlock of a holder that is still running.** If the holder's process still runs when its lock is forced out (or is stuck and later runs again), the window is the forced holder's whole remaining hold: from the moment its lock is removed until its own release is done. Inside it:
  - **Before its fence.** The holder finds the loss at its fence, commits nothing and runs its whole read-modify-write again under a fresh lock. Nothing is lost.
  - **Between its fence and its commit rename** (one read and one rename call, unless the holder stalls there). Its commit lands although its lock is gone. A newer writer that acquired the lock in the meantime and read the spawn-log commits too, and whichever of the two commits lands last drops the other's record, although that writer reported success. The board write in `pa dispatch` has the same window between its own fence and its rename: a board item can be lost there (the record stays in the spawn-log, and `pa board check` names the dispatch as not on the board).
  - **Its release.** The release moves whatever lock holds the name, by then a newer holder's, sees that the bytes are not its own and puts that lock back by exclusive create. If a third writer took the name in that gap, the put-back finds the name taken and the newer lock is dropped. The newer holder's fence then refuses its commit and it retries, unless it is already between its fence and its commit rename: then its commit lands while the third writer holds the lock, and the third writer's commit, from a read taken before it, can drop that record. A release killed after that move (`release:mismatch`, or `release:put-back-opened` without hard links) leaves the newer holder the same way, with an empty lock under the lock name in the second case.

  So inside this window guarantee 1 does not hold, and guarantee 2 does not hold either: the forced holder's release moves a newer live holder's lock.
- **A forced unlock inside a breaker's confirm-to-rename window.** `--force` does not stop for other breakers' claims. If it removes a dead lock that another breaker has confirmed and not yet renamed (one read and one rename call, unless that breaker stalls there), and a new holder publishes a lock in that gap, the breaker moves the new lock, sees that its bytes are not the ones it judged, and puts it back by exclusive create. If a third writer took the lock name before the put-back, the new lock is dropped, with the consequences of a forced holder's release above: the new holder's fence refuses its commit and it retries, unless it is already between its fence and its commit rename, where a record can be lost. A breaker killed after that move (`break:mismatch`, or `break:put-back-opened` without hard links) leaves the new holder the same way, with the lock name free in the first case and an empty lock under it in the second (a block until `unlock --force` clears it). A new holder still before its fence finds the loss there, commits nothing and runs its whole read-modify-write again under a fresh lock. One already between its fence and its commit rename commits although its lock is gone; a third writer that takes the lock name (in the second case, once that block is cleared) and reads the spawn-log before that commit lands commits too, and whichever of the two commits lands last drops the other's record. So inside this window guarantee 1 does not hold, and guarantee 2 does not hold either: the breaker moves a new live holder's lock.

A block that `pa dispatch unlock` clears is a block, not a residual against either guarantee: writers wait 10 s, fail with exit 2 naming the holder, and write nothing.

The windows are `LOCK_TIMEOUT_MS` 10000 ms (how long a writer waits before it fails) and `LOCK_STALE_AFTER_MS` 60000 ms (the stale window, used only by `pa board check` to report a held lock; no break and no sweep reads it). `timeoutMs` is an option seam, so tests shorten it.

## The lock file and the names beside it

The lock's content is one JSON object: `pid`, `hostname` (`os.hostname()`), `token` (24 random hex characters, one per hold) and `acquiredAt` (ISO time). A lock that does not parse to an object with an integer `pid` is *unreadable*. A parsed lock with no `hostname` (an earlier shape) is judged as this host's.

Messages show a lock's `hostname` and `acquiredAt`, and the agent name in a lock path, capped for display (`capForDisplay`): a value longer than 32 characters is shown as its first 29 characters and `...`. The cap is display only: `pa board check --json` keeps `lockPath` and `holder.hostname` whole, and every command a message tells the operator to run keeps the whole agent name.

| Name | What it is | Owner |
|---|---|---|
| `<lock>` | the lock | its `pid` and `hostname` |
| `<lock>.pending-<host>-<pid>-<attempt>` | the lock's content before it is published; without hard links it is removed as soon as `link()` fails | in the name |
| `<lock>.claim-<key>-<host>-<pid>-<attempt>` | a breaker's claim on one lock: `<key>` is 16 hex of the sha256 of that lock's bytes | in the name |
| `<lock>.broken-<host>-<pid>-<attempt>` | a lock moved away by a break, a release or an unlock, before it is checked and removed | in the name |
| `<spawn-log>.tmp-<host>-<pid>-<attempt>` | the commit's temp file | in the name |

`<host>` is 8 hex of the sha256 of the hostname. `<attempt>` is 24 random hex characters drawn for that one attempt, so no two attempts, in one process or in two, ever share a name, and a name one process removes can never be re-created by another. Each process removes only names it created in that attempt, and names whose recorded owner is provably dead (the sweep, below).

The earlier shapes are read too, so an instance that ran an earlier build is cleared: `<lock>.claim-<inode>` (a hard link or copy of a lock with no `hostname`: the owner is the `pid` in its content), `<lock>.claim-h<key>` (the owner is the `pid` and `hostname` in its content), `<lock>.broken-<pid>-<hex>` and `<spawn-log>.tmp-<pid>-<hex>` (no host: judged as this host's), and `<lock>.broken-<host>-<pid>-<time>-<hex>`. No current writer creates any of these names.

## The one automatic break

A writer that finds the lock held breaks it only when the lock parses, its holder is on this host (or names no host) and `process.kill(pid, 0)` fails with anything but `EPERM` (`EPERM` reads as alive). Nothing else is ever broken automatically: a live pid however old, a holder on another host, an empty or unreadable file, or a pid that has been reused (it answers, so it reads as alive). Age is never a reason.

## The steps

Each step name below is a test-only seam (`onStep`), awaited at that point; the shipped path never sets it.

**Acquire.**
1. Sweep (below).
2. Publish. Write the content to a pending file (`acquire:pending`), then `link()` it to the lock name, which fails with `EEXIST` when a lock exists, so the lock appears whole or not at all; then remove the pending file (`acquire:published`). Without hard links (any `link()` error other than `ENOENT` or `EEXIST`): remove the pending file, then `open(lock, 'wx')` (`acquire:opened`), write the content, close (`acquire:published`); for the rest of that hold no pending file is written. If that write or close fails, the handle is closed and nothing is unlinked: the file left is a parsed lock of this process (broken by the one automatic break once this process has exited) or an empty one (a block that `pa dispatch unlock --force` clears).
3. On `EEXIST`, read the lock (one handle: file time and bytes). If its holder is provably dead, break it, then go to 2 at once. Otherwise wait about 10 ms and go to 2. After `LOCK_TIMEOUT_MS`, fail with `SpawnLogLockError`, whose `holder` is the lock as last read: pid, hostname, `acquiredAt`, age and whether it is alive, dead, on another host or unreadable.

The filesystem class is learned per hold: `cls` starts null in every hold, a `link()` that succeeds makes it `links`, a `link()` error other than `ENOENT` or `EEXIST` makes it `nolinks`, and the next hold learns it again. Nothing is cached across holds; on a filesystem without hard links each hold makes one failing `link()` call before it opens the lock with `wx`. An `EEXIST` proves nothing, since a filesystem without hard links also refuses an existing name with `EEXIST`.

**Break** (a dead holder, and `pa dispatch unlock`).
1. `break:judged`. Claim: create `<lock>.claim-<key>-<host>-<pid>-<attempt>` for the bytes judged, exclusively, then list the folder. Another claim with the same key whose owner is provably dead is removed (its name is unique, so nothing can re-create it). Another claim with the same key whose owner is alive or on another host means another breaker is on this lock: remove this claim and wait. Of two breakers whose claims overlap, the one that lists later sees the other's claim, so at most one goes on (`break:claimed`).
2. Confirm: re-read the lock. It must have the bytes judged and still be provably dead (`unlock --force` checks the bytes only) (`break:confirmed`). A lock's bytes are its identity on both classes: every lock carries a random token, so its bytes name one lock.
3. Rename the lock to `<lock>.broken-<host>-<pid>-<attempt>` (`break:renamed`).
4. Check the moved file. When it has the bytes judged, remove it (`break:removed`). When it does not (a lock that `unlock --force` removed and a new holder replaced inside the confirm-to-rename window), put it back by exclusive create (`break:mismatch`): with hard links `link()` it to the lock name; without them `open(lock, 'wx')` (`break:put-back-opened`) and write its bytes. If the lock name is taken, the moved lock is dropped and its holder's fence finds the loss. Then remove the moved name (`break:put-back`).
5. Remove the claim.

**Hold and commit.** Read the spawn-log (`hold:read`), apply the change, write the temp file (`commit:temp-written`), run the fence (`commit:fenced`), rename the temp file over the spawn-log (`commit:renamed`). `pa dispatch` then patches the board while the lock is still held (`afterWrite`), and the board's own rename is preceded by the same fence: if the lock is lost there, the board write is skipped and the dispatch exits 2 naming the id (the record is in the spawn-log and `pa board check` names it). The spawn-log commit is never repeated after it lands. The board's own temp file lives in the board's folder and is not the lock's: a process killed inside the board write can leave it there, and the sweep does not touch it.

**Release.** Rename the holder's own lock to `<lock>.broken-<host>-<pid>-<attempt>` (`release:renamed`) and check the moved bytes. When they are the holder's own, remove the moved file (`release:removed`). When they are not (only after `unlock --force` and a new holder), put the moved lock back by exclusive create as a break does (`release:mismatch`, `release:put-back-opened` without hard links, `release:put-back`). A release never unlinks the lock name, so a release stalled for any time can only ever remove its own moved file.

**Sweep** (at the start of every acquire). Every name beside the lock in the shapes above is removed when its recorded owner (in its name, or in its content for the earlier claim shapes) is on this host and its pid is dead (`sweep:judged` is the seam between that judgement and the unlink). Nothing is swept by age. A leftover whose owner is alive or on another host is left alone. A leftover whose owner cannot be read (only the earlier claim shapes keep their owner in their content) is left alone too: no current writer reads those names, so it blocks nothing, and it can be deleted by hand.

**Unlock** (`pa dispatch unlock [--force] [--agent <name>]`). Read the lock and print its holder. No lock: exit 0. A holder that is provably dead: break it (the steps above). Any other holder: exit 2, naming the holder and `--force` with when to use it (only once the holder's process is gone or stuck, never while it is still running), and the lock is left byte for byte. `--force`: the holder is printed first, then the lock is broken with the bytes check only, and other breakers' claims do not stop it. The removal never unlinks a name it has not just proved is the lock it printed: if the lock changed since it was read, nothing is removed and unlock exits 2. Without `--force`, a live claim of another breaker on the same lock makes unlock wait up to a second and then exit 2 naming that claim's owner. Unlock sweeps nothing but the lock it judged.

**`pa board check`** reads the lock file (it never writes, breaks or sweeps) and reports a lock held for longer than the stale window as a problem (exit 2) whose line leads with `pa dispatch unlock` and the holder's pid, then its host, state, age and `acquiredAt`, and, for a holder that is not provably dead, when to use `--force`. The same line reaches `--hook-envelope`. Every variable part of it is bounded (a pid, the capped names, an age), so the line is at most 362 characters (346 for a seven-digit pid), and the envelope's 360-character cut can drop only the end of its `--force` sentence.

## The clock

No break and no sweep reads a clock: the one automatic break is a pid check, and the sweep is a pid check. Seven things read one, and none of them decides whether a lock is broken or a name is swept:

- **The 10 s acquisition timeout.** It is measured with this process's own wall clock (`Date.now()`) from the start of the acquire, so a skewed lock time never shortens or lengthens it. A step of this host's clock while the writer waits lengthens the wait (a step backwards) or shortens it (a step forwards).
- **The retry deadline of an update.** `updateSpawnLog` sets a deadline one lock timeout after its start, from `Date.now()`; after a lost lock it runs again only while that deadline has not passed, and each new hold gets the time left. A clock step changes it as it changes the acquisition timeout. It decides only how long a writer that was fenced out keeps retrying before it fails with exit 2.
- **The unlock busy-wait.** Without `--force`, while another breaker's live claim is on the lock, `removeSpawnLogLock` tries again for up to `UNLOCK_BUSY_WAIT_MS` (1000 ms), measured from `Date.now()`. A step backwards of the wall clock lengthens that wait and a step forwards shortens it; either way it decides only how long a non-force `pa dispatch unlock` waits before it exits 2 naming the claim's owner.
- **The lock's `acquiredAt`**, stamped from the holder's clock when it publishes the lock (the `now` seam, `Date.now()` in the shipped path). It is read only for the age below.
- **The claim's `at` stamp.** A claim's content records `at` (`new Date()`) when the claim is created. It is written for diagnosis, and nothing reads it to decide anything: a claim's owner is read from its name.
- **The age printed** in the exit-2 message and by `pa dispatch unlock`: the time since the later of `acquiredAt` and the lock's file time (for an unreadable lock, its file time), never below 0. A time skewed into the future reads as a lock just taken; a pair far in the past reads as old. The age is shown to the operator and decides nothing.
- **The stale-window line of `pa board check`**, from the same age. A lock whose times are in the future is not reported until its age passes the stale window; a lock whose times are both far in the past is reported at once. Either way the lock itself is never broken by it.

## States and crash points

Test files: `write` is `packages/core/test/spawn-log/write.test.mjs`, `crash` is `packages/core/test/spawn-log/lock-crash.test.mjs`, `dispatch` is `packages/cli/test/pa/dispatch-cli.test.mjs`, `unlock` is `packages/cli/test/pa/dispatch-unlock.test.mjs` and `board` is `packages/cli/test/pa/board-check.test.mjs`. `<class>` in a title is `hard links` or `no hard links`: the same test runs once per column, the second with `link()` failing with `ENOTSUP` (through the `link` seam in core tests, through a preload that makes `fs.promises.link` fail in tests of the real bin). "Kill" means the process kills itself with `SIGKILL` at that step through the `onStep` seam; each kill test then runs the next write at once, with no wait, and checks that the folder holds only the spawn-log and that no committed record is missing. The `crash` › mismatch tests run a real holder whose lock the killed breaker or releaser moved: the holder waits after its read, a contender commits, and the holder must find the loss at its fence, retry under a fresh lock and commit after the contender. G1 is guarantee 1, G2 guarantee 2.

| Row | Hard links available (APFS, ext4, NTFS) | Hard links unavailable (exFAT and other local volumes without them) |
|---|---|---|
| Free | Pending file, `link()` to the lock name, pending removed; the holder writes and releases. G1, G2. `write` › lock (hard links): a free lock is taken whole and released, nothing left beside the spawn-log | The pending file's `link()` fails and the pending file is removed; then `open wx` and the content written. G1, G2. `write` › lock (no hard links): a free lock is taken whole and released, nothing left beside the spawn-log |
| A live holder, short | The writer waits, then fails with `SpawnLogLockError` naming the holder; the lock is untouched. G2. `write` › lock (hard links): a lock held by a live process times out with SpawnLogLockError naming the holder and is left in place | Same. G2. `write` › lock (no hard links): a lock held by a live process times out with SpawnLogLockError naming the holder and is left in place |
| A live holder of any age past the stale window | Never broken. A writer that arrives while the holder still works waits and both records survive; one that waits out the timeout exits 2 from `pa dispatch` naming the holder and `pa dispatch unlock --force`, writing nothing. G1, G2. `write` › lock (hard links): a live holder of any age is never broken: past the stale window, at 11 minutes and at one hour; `crash` › fence (hard links): a live holder stalled past the stale window keeps its lock; the contender waits and both records survive; `unlock` › dispatch open (hard links): a held lock makes open exit 2 after the timeout with one line naming the holder and pa dispatch unlock, and nothing is written; and `unlock` › dispatch open through the real bin against a live holder exits 2 after the 10 s timeout with the holder line | Same. G1, G2. `write` › lock (no hard links): a live holder of any age is never broken: past the stale window, at 11 minutes and at one hour; `crash` › fence (no hard links): a live holder stalled past the stale window keeps its lock; the contender waits and both records survive; and `unlock` › dispatch open (no hard links): a held lock makes open exit 2 after the timeout with one line naming the holder and pa dispatch unlock, and nothing is written |
| A dead holder on this host (also a lock with no `hostname`) | Broken at once: claim, confirm, rename, check, remove. G1, G2. `write` › lock (hard links): a lock left by a dead process on this host is broken at once, with or without a hostname | Same. G1, G2. `write` › lock (no hard links): a lock left by a dead process on this host is broken at once, with or without a hostname |
| An empty or unreadable lock, of any age | Never broken: a block. Writers exit 2 after the timeout naming it and `pa dispatch unlock --force`; our own writer never leaves one on this class (the lock is published whole). `write` › lock (hard links): an empty or unreadable lock is never broken, at any age, and the timeout names it; and `unlock` › unlock (hard links): an empty or unreadable lock is refused without --force and removed with --force | Never broken: a block, as in the other column. On this class our own writer leaves one only when it is killed between `open wx` and the write (the kill rows). `write` › lock (no hard links): an empty or unreadable lock is never broken, at any age, and the timeout names it; and `unlock` › unlock (no hard links): an empty or unreadable lock is refused without --force and removed with --force |
| A holder on another host, of any age | Never broken and its pid never checked here: a block until its holder releases it, or `unlock --force`. G2. `write` › lock (hard links): a holder on another host is never broken, at any age, and its pid is never checked here | Same. G2. `write` › lock (no hard links): a holder on another host is never broken, at any age, and its pid is never checked here |
| A holder whose pid is reused, of any age | The pid answers, so it reads as alive: never broken; a block cleared by `unlock --force`. G2. `write` › lock (hard links): a reused pid reads as a live holder and is never broken, at any age | Same. G2. `write` › lock (no hard links): a reused pid reads as a live holder and is never broken, at any age |
| A lock whose times are skewed (the clock) | A skewed `acquiredAt` or file time never breaks a live lock and never stops the break of a dead one; the age printed reads from the later of the two. G2. `write` › lock (hard links): a lock whose acquiredAt and file time are skewed into the future or far into the past is never broken, and its age reads from the later of the two; and `board` › kind lock-held: lock times skewed into the future read as just taken, and a pair far in the past reads as old | Same. G2. `write` › lock (no hard links): a lock whose acquiredAt and file time are skewed into the future or far into the past is never broken, and its age reads from the later of the two |
| Two breakers of one dead-holder lock | Both claim; the one that lists later sees the other's claim and waits; one breaks, then both enter in turn, and nothing is left. G1, G2. `write` › lock (hard links): two breakers at once against a dead-holder lock both run, never together, and leave no lock, claim or broken file | Same. G1, G2. `write` › lock (no hard links): two breakers at once against a dead-holder lock both run, never together, and leave no lock, claim or broken file |
| A breaker that judged a lock since replaced | Its claim goes through, its confirm sees other bytes, and the fresh lock is never moved (the test also checks its inode). G2. `write` › lock (hard links): a breaker that judged a dead lock just before another broke it and took a fresh lock leaves the fresh lock alone | Its confirm sees other bytes; the fresh lock is never moved. G2. `write` › lock (no hard links): a breaker that judged a dead lock just before another broke it and took a fresh lock leaves the fresh lock alone |
| A breaker stalled past any window holding its claim | Every other breaker of that lock sees the live claim and waits: a block. Writers exit 2 naming the dead holder; `unlock` without `--force` waits a second and refuses, naming the claim's owner; when the breaker resumes it finishes alone. G1, G2. `crash` › stall (hard links): a breaker stalled past any window after its claim keeps every other breaker of that lock waiting; they time out naming the dead holder, unlock refuses while the claim is live, and the breaker finishes alone | Same. G1, G2. `crash` › stall (no hard links): a breaker stalled past any window after its claim keeps every other breaker of that lock waiting; they time out naming the dead holder, unlock refuses while the claim is live, and the breaker finishes alone |
| Eight writers at once against a dead-holder lock, 25 trials | Every exit 0, ids distinct, every record and board item present, nothing left. G1, G2. `dispatch` › dispatch open from 8 processes against a stale lock, 25 trials: distinct ids, nothing lost, every exit 0 | Same. G1, G2. `dispatch` › dispatch open with link() failing (ENOTSUP) from 8 processes against a stale lock, 25 trials: distinct ids, nothing lost, every exit 0 |
| `link()` fails with a code other than `ENOENT` or `EEXIST` | A filesystem that refuses `link()` is the other column's from that call on, for that hold: no such error reaches `pa dispatch`. `crash` › no hard links: link() failing with EPERM falls back; two writes in a row and eight at once, nothing lost, nothing left; and `dispatch` › dispatch open with link() failing (EPERM): two in a row and eight at once, then eight against a stale lock: every exit 0, nothing lost, nothing left | `EPERM`, `EOPNOTSUPP` and `ENOTSUP` all fall back. `crash` › no hard links: link() failing with EOPNOTSUPP falls back; two writes in a row and eight at once, nothing lost, nothing left (and the same with ENOTSUP); and `dispatch` › dispatch open with link() failing (EOPNOTSUPP): two in a row and eight at once, then eight against a stale lock: every exit 0, nothing lost, nothing left (and the same with ENOTSUP) |
| A publish whose write or close fails after its exclusive step | The pending file is the writer's own and is removed; nothing else is unlinked, so a lock another process holds stays. G2. `write` › lock (hard links): a publish that fails after its exclusive step unlinks nothing but its own pending file, and a replacement lock stays | The handle is closed and nothing is unlinked, by name or by inode: a replacement lock stays byte for byte. The file left is the one automatic break's or `unlock`'s. G2. `write` › lock (no hard links): a publish that fails after its exclusive step unlinks nothing but its own pending file, and a replacement lock stays |
| A releaser stalled past any window | Its lock is already moved to its own name, so others acquire, commit and release meanwhile; on resuming it removes only its own moved file. G1, G2. `crash` › stall (hard links): a releaser stalled past any window after moving its lock removes only its own moved file; other writers acquire, commit and release meanwhile, and every record survives | Same. G1, G2. `crash` › stall (no hard links): a releaser stalled past any window after moving its lock removes only its own moved file; other writers acquire, commit and release meanwhile, and every record survives |
| A releaser resuming inside another holder's fence-to-rename window | The holder's lock is untouched, its commit lands, and a third writer commits after it. G1, G2. `crash` › stall (hard links): a releaser resuming while another holder is between its fence and its rename leaves that lock in place, and no record is lost | Same. G1, G2. `crash` › stall (no hard links): a releaser resuming while another holder is between its fence and its rename leaves that lock in place, and no record is lost |
| A sweeper stalled past any window between its judgement and its unlink | It removes only the dead owner's leftover it judged (a unique name nothing can re-create); a live owner's leftover and every live lock stay. G1, G2. `crash` › stall (hard links): a sweeper stalled past any window between its judgement and its unlink removes only the dead owner's leftover, and every record survives | Same. G1, G2. `crash` › stall (no hard links): a sweeper stalled past any window between its judgement and its unlink removes only the dead owner's leftover, and every record survives |
| `unlock` with no lock | Exit 0, says so, writes nothing. `unlock` › unlock (hard links): with no lock, unlock exits 0, says so and writes nothing; and `unlock` › unlock: a usage error exits 1 and touches nothing | Same. `unlock` › unlock (no hard links): with no lock, unlock exits 0, says so and writes nothing |
| `unlock` on a dead holder | The holder is printed, the lock broken by the break's steps, exit 0, and the next open exits 0. G1. `unlock` › unlock (hard links): a lock whose holder is provably dead is removed, unlock exits 0 naming it, and the next open exits 0 | Same. G1. `unlock` › unlock (no hard links): a lock whose holder is provably dead is removed, unlock exits 0 naming it, and the next open exits 0 |
| `unlock` on a live holder, a holder on another host or a reused pid | Refused: exit 2, the holder and `--force` named, the lock byte for byte. G2. `unlock` › unlock (hard links): a live holder, a holder on another host and a reused pid are refused: exit 2, the holder and --force named, the lock byte for byte | Same. G2. `unlock` › unlock (no hard links): a live holder, a holder on another host and a reused pid are refused: exit 2, the holder and --force named, the lock byte for byte |
| `unlock --force` | The holder is printed, then the lock removed (exit 0). A live holder forced out before its fence finds the loss there, commits nothing, retries under a fresh lock, and both records survive. G1 and G2 residual: a forced unlock of a holder that is still running (its whole remaining hold). `unlock` › unlock (hard links): --force prints the holder, then removes the lock; a live holder stalled after its read is fenced out, retries, and both records survive; and `crash` › fence (hard links): a holder whose lock is removed by unlock --force while it holds is fenced out, retries under a fresh lock, and both records survive | Same. G1 and G2 residual: a forced unlock of a holder that is still running. `unlock` › unlock (no hard links): --force prints the holder, then removes the lock; a live holder stalled after its read is fenced out, retries, and both records survive; and `crash` › fence (no hard links): a holder whose lock is removed by unlock --force while it holds is fenced out, retries under a fresh lock, and both records survive |
| A lock lost under a holder (fence and release) | The holder commits nothing, and its release puts the newer lock back. G1. `crash` › fence (hard links): a lost lock never commits and its release leaves the newer lock; and `write` › lock (hard links): release puts back a lock that replaced the holder's own file | Same; the put-back is a copy, so the lock's inode changes and the fence goes by the token. G1. `crash` › fence (no hard links): a lost lock never commits and its release leaves the newer lock; and `write` › lock (no hard links): release puts back a lock that replaced the holder's own file |
| `pa board check` with a held lock | Younger than the stale window: no line. Older: a problem line (exit 2) that leads with `pa dispatch unlock` and the holder's pid, in the plain, `--json` and `--hook-envelope` output (the envelope still exits 0), whole under the envelope's cut with a 240-character agent name and a 255-character host. Read-only. `board` › kind lock-held: a spawn-log lock held longer than the stale window is a problem naming the holder and pa dispatch unlock, in the plain, --json and --hook-envelope output; inside the window it is not; `board` › pa bin (hard links): board check reports a held lock older than the stale window and stays read-only; `board` › pa bin: with a 240-character agent name, a lock held past the stale window by a live holder on this host or by a 255-character host gives one lock-held line that leads with pa dispatch unlock and the pid, caps the host, and is whole in the plain, --json and --hook-envelope output; and `unlock` › a 240-character agent name and a 255-character hostname: the exit-2 line of open and unlock's holder and refusal lines lead with the lock and the pid, cap the host and the agent in the lock path, and keep the whole agent in every command | Same: the check only reads the lock file. `board` › pa bin (no hard links): board check reports a held lock older than the stale window and stays read-only |
| The board write in `pa dispatch` after the lock is lost | The board write is skipped and the dispatch exits 2 naming the id; the record stays in the spawn-log. G1. `dispatch` › dispatch (hard links): a lock lost after the spawn-log commit skips the board write and exits 2 naming the id | Same; the fence reads the token on both classes. G1. `dispatch` › dispatch (no hard links): a lock lost after the spawn-log commit skips the board write and exits 2 naming the id |
| Leftovers in the earlier shapes with a dead owner | The first earlier shape (a lock with no `hostname`, an inode-named claim, a broken and a temp file with no host) and the second (a hash claim, a pending, a broken and a temp file with host and pid) are cleared at the first acquire, with no wait, and the next write loses nothing. G1. `write` › lock (hard links): leftovers in the first earlier shape with a dead owner (a lock with no hostname, an inode-named claim, a broken and a temp file with no host) are cleared at the first acquire, with no wait; and `write` › lock (hard links): leftovers in the second earlier shape with a dead owner (a hash claim, a pending, a broken and a temp file with host and pid) are cleared at the first acquire, with no wait | Same. G1. `write` › lock (no hard links): leftovers in the first earlier shape with a dead owner (a lock with no hostname, an inode-named claim, a broken and a temp file with no host) are cleared at the first acquire, with no wait; and `write` › lock (no hard links): leftovers in the second earlier shape with a dead owner (a hash claim, a pending, a broken and a temp file with host and pid) are cleared at the first acquire, with no wait |
| Leftovers whose owner is alive, on another host or unreadable | Left alone, in every shape; they block nothing and the next write loses nothing. G1, G2. `write` › lock (hard links): leftovers whose owner is alive, on another host or unreadable are left alone, and the next write loses nothing | Same. G1, G2. `write` › lock (no hard links): leftovers whose owner is alive, on another host or unreadable are left alone, and the next write loses nothing |
| Names unique per attempt | Claim, broken, pending and temp names carry a fresh 24-hex attempt token; two attempts never share one. G2. `write` › lock (hard links): claims, broken files, pending files and temp files carry the attempt's own token, so two attempts never share a name | Same. G2. `write` › lock (no hard links): claims, broken files, pending files and temp files carry the attempt's own token, so two attempts never share a name |
| Kill in acquire, after the pending file is written | Pending file left, owner dead: swept at once. G1. `crash` › kill (hard links) at acquire:pending: the next write clears what is left at once, and no committed record is lost | Same: each hold writes its pending file before its first `link()` fails. G1. `crash` › kill (no hard links) at acquire:pending: the next write clears what is left at once, and no committed record is lost |
| Kill in acquire, after `open wx` and before the write | No such step: the lock appears whole by `link()`. `write` › lock (hard links): a free lock is taken whole and released, nothing left beside the spawn-log | An empty lock is left: a block. The next write exits with the holder "empty or unreadable", `unlock --force` clears it, and the write after that goes through. G1. `crash` › kill (no hard links) at acquire:opened: the empty lock left blocks the next write until unlock --force clears it, and no committed record is lost |
| Kill in acquire, after the lock is published | A lock with a dead holder: broken at once (a pending file, if left, swept). G1. `crash` › kill (hard links) at acquire:published: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at acquire:published: the next write clears what is left at once, and no committed record is lost |
| Kill in the sweep, between its judgement and its unlink | The judged leftover (owner dead) is still there: swept at once by the next write. G1. `crash` › kill (hard links) at sweep:judged: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at sweep:judged: the next write clears what is left at once, and no committed record is lost |
| Kill in a break, after the judgement | The dead lock is still there: broken at once by the next write. G1. `crash` › kill (hard links) at break:judged: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at break:judged: the next write clears what is left at once, and no committed record is lost |
| Kill in a break, after the claim | The claim (owner dead: swept) and the dead lock (broken). G1. `crash` › kill (hard links) at break:claimed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at break:claimed: the next write clears what is left at once, and no committed record is lost |
| Kill in a break, after the confirm | The claim (swept) and the dead lock (broken). G1. `crash` › kill (hard links) at break:confirmed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at break:confirmed: the next write clears what is left at once, and no committed record is lost |
| Kill in a break, after the rename | The moved dead lock and the claim (owner dead: swept); the lock name is free. G1. `crash` › kill (hard links) at break:renamed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at break:renamed: the next write clears what is left at once, and no committed record is lost |
| Kill in a break, after the moved lock is removed | The claim: swept. G1. `crash` › kill (hard links) at break:removed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at break:removed: the next write clears what is left at once, and no committed record is lost |
| Kill in a break that moved a newer lock, before the put-back | Reached only after `unlock --force` and a new holder inside the breaker's confirm-to-rename window. The lock name is free; the moved lock and the claim are swept; the newer lock's holder has lost it. A holder still before its fence is refused there and retries (the order the `crash` › mismatch test runs); one already between its fence and its commit rename commits, and a record can be lost (the confirm-to-rename residual above). G1 and G2 residual: a forced unlock inside a breaker's confirm-to-rename window. `crash` › kill (hard links) at break:mismatch: the next write clears what is left at once, and no committed record is lost; and `crash` › mismatch (hard links) at break:mismatch: a breaker killed after it moved a live holder's lock; the holder is fenced out, retries under a fresh lock and commits after a contender, and every record survives once | Same. G1 and G2 residual: the same. `crash` › kill (no hard links) at break:mismatch: the next write clears what is left at once, and no committed record is lost; and `crash` › mismatch (no hard links) at break:mismatch: a breaker killed after it moved a live holder's lock; the holder is fenced out, retries under a fresh lock and commits after a contender, and every record survives once |
| Kill in a break's put-back, after `open wx` and before the write | No such step: the put-back is a `link()`. `crash` › kill (hard links) at break:put-back: the next write clears what is left at once, and no committed record is lost | An empty lock is left under the lock name (a block cleared by `unlock --force`); the moved lock and the claim are swept; the newer lock's holder has lost it. A holder still before its fence is refused there and retries (the order the `crash` › mismatch test runs); one already between its fence and its commit rename commits, and once the block is cleared a record can be lost (the confirm-to-rename residual above). G1 and G2 residual: the same as the row above. `crash` › kill (no hard links) at break:put-back-opened: the empty lock left blocks the next write until unlock --force clears it, and no committed record is lost; and `crash` › mismatch (no hard links) at break:put-back-opened: a breaker killed after it moved a live holder's lock leaves an empty lock that unlock --force clears; the holder is fenced out, retries under a fresh lock and commits after a contender, and every record survives once |
| Kill in a break, after the put-back | The newer lock is back under the lock name with its holder's token; the moved name and the claim are swept. G1, G2. `crash` › kill (hard links) at break:put-back: the next write clears what is left at once, and no committed record is lost | Same; the put-back is a copy. G1, G2. `crash` › kill (no hard links) at break:put-back: the next write clears what is left at once, and no committed record is lost |
| Kill while holding, after the spawn-log is read | A dead holder: broken at once; nothing was committed. G1. `crash` › kill (hard links) at hold:read: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at hold:read: the next write clears what is left at once, and no committed record is lost |
| Kill in the commit, after the temp file is written | The temp file (owner dead: swept) and the lock (dead holder: broken); the spawn-log is unchanged. G1. `crash` › kill (hard links) at commit:temp-written: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at commit:temp-written: the next write clears what is left at once, and no committed record is lost |
| Kill in the commit, after the fence | Same as the row above. G1. `crash` › kill (hard links) at commit:fenced: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at commit:fenced: the next write clears what is left at once, and no committed record is lost |
| Kill after the commit rename, before release | The record is in the spawn-log; the lock (dead holder) is broken at once. G1. `crash` › kill (hard links) at commit:renamed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at commit:renamed: the next write clears what is left at once, and no committed record is lost |
| Kill in release, after the rename | The holder's own moved lock (owner dead): swept; the lock name is free. G1. `crash` › kill (hard links) at release:renamed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at release:renamed: the next write clears what is left at once, and no committed record is lost |
| Kill in release, after the moved lock is removed | Nothing left. G1. `crash` › kill (hard links) at release:removed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) at release:removed: the next write clears what is left at once, and no committed record is lost |
| Kill in a release that moved a newer lock, before the put-back | Reached only after `unlock --force` removed the releaser's lock while it still ran, and a new holder. The lock name is free; the moved lock is swept; the newer lock's holder has lost it. A holder still before its fence is refused there and retries (the order the `crash` › mismatch test runs); one already between its fence and its commit rename commits, and a record can be lost (the forced holder's residual above). G1 and G2 residual: a forced unlock of a holder that is still running. `crash` › kill (hard links) at release:mismatch: the next write clears what is left at once, and no committed record is lost; and `crash` › mismatch (hard links) at release:mismatch: a releaser killed after it moved a live holder's lock; the holder is fenced out, retries under a fresh lock and commits after a contender, and every record survives once | Same. G1 and G2 residual: the same. `crash` › kill (no hard links) at release:mismatch: the next write clears what is left at once, and no committed record is lost; and `crash` › mismatch (no hard links) at release:mismatch: a releaser killed after it moved a live holder's lock; the holder is fenced out, retries under a fresh lock and commits after a contender, and every record survives once |
| Kill in a release's put-back, after `open wx` and before the write | No such step: the put-back is a `link()`. `crash` › kill (hard links) at release:put-back: the next write clears what is left at once, and no committed record is lost | An empty lock is left (a block cleared by `unlock --force`); the moved lock is swept; the newer lock's holder has lost it. A holder still before its fence is refused there and retries (the order the `crash` › mismatch test runs); one already between its fence and its commit rename commits, and once the block is cleared a record can be lost (the forced holder's residual above). G1 and G2 residual: the same as the row above. `crash` › kill (no hard links) at release:put-back-opened: the empty lock left blocks the next write until unlock --force clears it, and no committed record is lost; and `crash` › mismatch (no hard links) at release:put-back-opened: a releaser killed after it moved a live holder's lock leaves an empty lock that unlock --force clears; the holder is fenced out, retries under a fresh lock and commits after a contender, and every record survives once |
| Kill in a release, after the put-back | The newer lock is back with its holder's token; the moved name is swept. G1, G2. `crash` › kill (hard links) at release:put-back: the next write clears what is left at once, and no committed record is lost | Same; the put-back is a copy. G1, G2. `crash` › kill (no hard links) at release:put-back: the next write clears what is left at once, and no committed record is lost |
| Kill in `unlock` | Unlock runs the break's steps, so each break row applies to it. After the rename: the moved lock and unlock's claim (owner dead) are swept and the lock name is free. G1. `crash` › kill (hard links) in unlock at break:renamed: the next write clears what is left at once, and no committed record is lost | Same. G1. `crash` › kill (no hard links) in unlock at break:renamed: the next write clears what is left at once, and no committed record is lost |

The table has 49 rows and 98 cells, and each of the 98 cites the test or tests that prove it; no cell rests on a probe.

Inside the envelope, every residual against G1 or G2 starts with `unlock --force`: the forced holder's whole remaining hold, when its process is still running, and a forced unlock inside another breaker's confirm-to-rename window. No cell has a residual without `--force`.

## The core lock

`acquireLock` in `packages/core/src/lock/lock.mjs` (used by the launch runner and the library sync) is a separate lock with its own known race: two processes can hold it at once. It should adopt this primitive later: the four-field lock file, the one automatic break, the per-attempt names, the claim, the fence, the sweep of provably dead owners and the unlock. Until then it is not covered by this page.