# v0.3.9-alpha.4 Release Notes

## Version

`v0.3.9-alpha.4`

## Dist Tag

`alpha`

During the alpha pilot, `latest` may be synchronized to the current alpha build to avoid stale default installs. This is not a stable support claim.

## Summary

This prerelease packages eval tooling, scorer, fixture metadata, and purity-guard improvements after `persona-harness@0.3.9-alpha.3`.

This is a tooling/scorer/purity-guard release. It does not include a fresh actual eval rerun after the final toolchain-aware scorer and policy-marker changes. It does not claim PH ON passed under the new toolchain-aware policy, does not claim the v0.4 matrix improved or passed, and does not reinterpret old v0.4.1 or unstamped results as new outcome evidence.

Persona Harness remains an AI coding workflow rail + evidence + continuation harness. It is not a generated app product-quality certification, Java Clean Code guarantee, AST/linter, or enforcement engine.

## What Changed

### Workspace isolation and baseline purity

- `1aa55e6 fix(eval): isolate onoff workspaces`
  - Moves the default output root outside the repo temp area.
  - Adds preflight detection for ambient `AGENTS.md`, `CLAUDE.md`, `.persona`, and `.opencode` influence.
  - Adds a baseline post-run purity guard.
  - Contaminated baseline results decide INCONCLUSIVE rather than becoming clean ON/OFF evidence.

### Toolchain-aware scoring

- `eafd0bf fix(eval): score generated toolchains`
  - Detects generated Gradle, Maven, Python, and Unknown toolchains.
  - Runs matching build/test scoring instead of forcing every generated output through Gradle metrics.
  - Prevents Maven/Python generated outputs from being unfairly counted as Gradle failures.

### Policy/scorer marker and aggregate separation

- `3aeaa1b fix(eval): stamp toolchain-aware policy`
  - Stamps toolchain-aware policy/scorer state on fresh and replay results.
  - Records generated `toolchain` and fixture `fixtureStackToolchain`.
  - Separates external outcome from stack/toolchain mismatch reporting.
- `51f1622 test(eval): cover toolchain aggregate separation`
  - Covers separated toolchain aggregates.
  - Keeps `stackToolchainMatchRate` diagnostic-only.

### Fixture metadata

- `0a05e6d fix(eval): align fixture toolchain types`
  - Aligns fixture metadata with the final toolchain policy.
- Fixture stack/toolchain metadata is confirmed:
  - `backend-api-no-stack`: free-stack.
  - `ambiguous-idea-first`: free-stack.
  - `multi-step-backend`: `java-spring-gradle-pinned`, stress/continuation.
  - `multi-step-backend-small`: `java-spring-gradle-pinned`, reduced paired fixture.
- Full `multi-step-backend` remains a stress/continuation fixture, not a simple single-turn matrix row.
- `multi-step-backend-small` remains the reduced single-turn paired fixture.

### Workflow backlog repair and scope guidance

- `212046f fix(cli): repair archived ticket backlog mismatch`
  - Diagnoses history-only pending tickets.
  - Allows explicit `workflow archive <ticket>` repair.
  - Keeps the finish gate strict.
- Related scope-continuation work remains part of the release context:
  - `1e7a97c docs(skills): guide oversized backlog continuation`.
  - `5bcd4dd fix(cli): clarify ticket scope continuation`.
  - `e6e5f5e feat(eval): add reduced multi-step fixture`.

## Eval Outcome Boundary

- No fresh actual eval outcome is claimed after the final toolchain-aware scorer/marker changes.
- The SIGINT/interrupted run after scorer work produced no `results.json` and is not evidence.
- Old `external-primary-v0.4.1` or unstamped results are not rewritten. If evaluated under the new policy, they are INCONCLUSIVE rather than retroactive PASS/FAIL evidence.
- `stackToolchainMatchRate` is diagnostic-only. It is not a hard gate, generated app quality proof, or product value proof.
- Full v0.4 matrix status is unchanged: it still requires a fresh toolchain-aware run before any new outcome conclusion.

## Verification

Release-prep verification for this worktree:

```bash
npm test
npm run typecheck
npm run build
npm run check:docs
npm pack --dry-run
git diff --check
```

Observed result on 2026-06-28:

- `npm test`: passed, 64 files / 428 tests.
- `npm run typecheck`: passed.
- `npm run build`: passed.
- `npm run check:docs`: passed.
- `npm pack --dry-run`: passed for `persona-harness@0.3.9-alpha.4`.
  - filename: `persona-harness-0.3.9-alpha.4.tgz`;
  - package size: `368.6 kB`;
  - unpacked size: `1.5 MB`;
  - shasum: `0df5a3e2fc54ccc6d00dc0ab75e3d82b8854f315`;
  - total files: `359`.
- `git diff --check`: passed.

## Published Registry Surface Smoke

`persona-harness@0.3.9-alpha.4` exists on the npm registry.

Registry facts observed after publish:

- version: `0.3.9-alpha.4`;
- gitHead: `8080699a1029e2740d055cf70af8c4dbc9508813`;
- dist.shasum: `ccd638553e52fe70a194578653ac1bb84b1eec1a`;
- dist-tags: `alpha=0.3.9-alpha.4`, `latest=0.3.9-alpha.3`.

Git facts observed after publish:

- HEAD: `8080699a1029e2740d055cf70af8c4dbc9508813`;
- local tag `v0.3.9-alpha.4` points at HEAD;
- origin tag `refs/tags/v0.3.9-alpha.4` points at the same HEAD.

Publish note:

- `npm publish --tag alpha` returned EOTP text, but immediate registry verification showed `persona-harness@0.3.9-alpha.4` exists and the `alpha` tag points to it.
- Registry state is treated as the source of truth.
- Do not republish the same immutable version.
- `latest` was not moved and remains `0.3.9-alpha.3`.

Registry install smoke:

- Clean temp project: `/private/tmp/persona-alpha4-smoke-oCXdPe`.
- `npm install -D persona-harness@alpha` installed `persona-harness@0.3.9-alpha.4`.
- `npx ph init --default backend`: PASS.
- `npx ph doctor`: PASS, Runtime readiness observed.
- `npx ph observe --json` on a tiny Java service fixture: PASS, finding schema keys present.

Smoke boundary:

- This is registry install/package surface smoke only.
- No actual eval rerun was performed.
- No generated app/product quality certification is made.
- `latest=0.3.9-alpha.3` is recorded as observed dist-tag state, not as a smoke failure.

## Post-Release Eval Outcome Evidence

QA ran a clean post-alpha4 eval after publish. Only the first fixture, `backend-api-no-stack`, ran; remaining fixtures were stopped after a valid FAIL.

Source and policy:

- PH install: `npm install -D persona-harness@0.3.9-alpha.4`.
- Policy: `external-primary-toolchain-v0.4.2`.
- Scorer: `generated-toolchain-v1`.
- Output root: outside the repo.
- Original results: `/var/folders/z8/4909cvgx53n79fj88q94nd200000gn/T/persona-harness-eval-runs/2026-06-28T034057112Z/results.json`.
- Replay results: `/var/folders/z8/4909cvgx53n79fj88q94nd200000gn/T/persona-harness-eval-runs/2026-06-28T044041297Z/results.json`.

Decision:

- Original decide: FAIL.
- Replay decide: FAIL.
- Baseline purity: PASS.
- PH ON instrumentation: valid.

Aggregate, matched in original and replay:

| condition | build | test | runtime | stack | workflow | failures |
| --- | --- | --- | --- | --- | --- | --- |
| plain | 100% | 100% | 0% | 0% | n/a | 4 |
| claude | 100% | 100% | 0% | 0% | n/a | 4 |
| agents | 100% | 100% | 0% | 0% | n/a | 4 |
| ph-on | 100% | 100% | 50% | 50% | 0% | 6 |

PH ON run-level notes:

- r1: Java/Gradle build/test/runtime PASS, but workflow finish FAIL with provider-limit/workflow-dead-end labels.
- r2: provider limit, incomplete/unknown output, runtime FAIL, workflow FAIL.

Interpretation:

- This is valid eval outcome evidence for one fixture under the alpha4 toolchain-aware scorer.
- It is not generated app quality certification.
- It is not a full v0.4 matrix result.
- It does not support a broad PH superiority claim.
- It blocks continuing the remaining fixtures until the workflow/provider cause is triaged.
- Alpha4 as a tooling/scorer/purity-guard release remains valid; this post-release eval evidence is separate from release-prep claims.

## Provider/Tool Completion Semantics

Research accepted `80943d2 fix(eval): separate provider completion outcomes` as a methodology fix.

What changed:

- Fresh results include `completionSemanticsVersion: provider-tool-completion-v1`.
- Generated app external metrics are separated from operational feasibility metrics.
- Run-level `providerToolCompletion` is recorded.
- Aggregates include `completionWithinBudgetRate`, `providerLimitRate`, `finishWithinBudgetRate`, and `operationalFailureModeTotal`.
- Provider incomplete + unknown workspace is reported as runtime NOT RUN, not fake runtime FAIL.
- Marker-less old/post-alpha4 results are INCONCLUSIVE under the new semantics. Existing verdicts are not retroactively changed.

Verification reported by QA:

- focused tests: PASS.
- full tests: PASS.
- `npm run typecheck`: PASS.
- `npm run build`: PASS.

Boundary:

- No actual eval rerun happened for this semantics change.
- No new eval outcome claim is made.
- Provider-limited/TIMED_OUT is an operational failure, not generated app quality failure.
- Scorable-only external metrics must not be highlighted without operational rates.
- Provider failures are not automatically treated as infra noise or INCONCLUSIVE.
- No generated app quality certification is made.

## Provider/Tool Completion v1 Fresh Rerun

QA reran `backend-api-no-stack` with the provider/tool completion semantics. Research classified the result as a valid eval FAIL, specifically operational-blocked / provider-tool-completion failure.

Source and markers:

- Fixture: `backend-api-no-stack`.
- Original results: `/var/folders/z8/4909cvgx53n79fj88q94nd200000gn/T/persona-harness-eval-runs/2026-06-28T053933590Z/results.json`.
- Replay results: `/var/folders/z8/4909cvgx53n79fj88q94nd200000gn/T/persona-harness-eval-runs/2026-06-28T063940663Z/results.json`.
- Policy: `external-primary-toolchain-v0.4.2`.
- Scorer: `generated-toolchain-v1`.
- Completion semantics: `provider-tool-completion-v1`.
- Replay parser fix: `0276062 fix(eval): preserve replay provider completion`.

Decision:

- Original decide: FAIL.
- Replay decide: FAIL.
- Failure subtype: operational-blocked / provider-tool-completion failure.

PH ON metrics:

- App external outcome: build/test/runtime `100%`, external failures `0`.
- Operational outcome: completion `0%`, providerLimit `100%`, finishWithinBudget `0%`, operational failures `3`.
- r1: app build/test/runtime PASS, then provider timeout and workflow incomplete.
- r2: app build/test/runtime/workflow PASS, but provider timeout remained operationally incomplete.

Baseline summary:

- plain/claude completed provider work but runtime failed.
- agents was mixed.

Boundary:

- Do not call this PH ON PASS because operational feasibility failed.
- Do not call provider timeout a generated app runtime/build failure.
- Do not hide provider timeout by showing scorable-only app metrics.
- This is not generated app quality certification.
- This is not a full matrix result, and matrix continuation remains blocked.
- Old results remain unchanged.

Next work:

- Reduce workflow finalization cost, define finalization-only continuation policy, or adjust timeout budget policy.
- Rerun the same fixture before continuing the matrix.

## Finalization Triage

CLI triaged the provider-tool-completion rerun without a fresh eval rerun.

Commit:

- `63dc661 fix(cli): ignore recovered verification notes`.

Findings:

- r1 bottleneck: app build/test/runtime PASS, tickets/reports filled, role coverage OK.
- r1 remaining blocker: missing `npx ph bearshell` final verification evidence in filled reports.
- CLI false positive fixed: report text such as "initial failure later recovered" was misread as current compile/test failure.
- Built CLI r1 smoke now reports verification failure `no failed verification recorded`.
- r2: provider TIMED_OUT but workflow PASS.
- Finish gate false negative was ruled out.
- Finish gate was not weakened.

Boundary:

- No fresh eval rerun happened.
- No generated app quality certification is made.
- This does not change the prior provider-tool-completion-v1 rerun verdict.

Next work:

- Run a QA/runner finalization-only continuation pass.
- Count that continuation separately from single-turn completion.

## Known Gaps

- No complete post-release eval matrix has been performed after the final toolchain-aware scorer/marker changes.
- Prior v0.4.1/unstamped eval results remain historical and diagnostic. They are not alpha4 outcome evidence.
- Toolchain-aware scoring improves measurement fairness, but it is not product-quality certification.
- Workspace isolation and baseline purity guards reduce contamination risk, but clean future reruns are still required before matrix conclusions.
- Registry alpha4 install surface passed, and one post-release `backend-api-no-stack` eval produced a valid FAIL; no generated-app quality certification was performed.
- Post-release `backend-api-no-stack` toolchain-aware eval is a valid one-fixture FAIL and blocks continuing the remaining fixtures until workflow/provider cause is triaged.
- Provider/tool completion v1 fresh rerun also FAILed operationally: PH ON app metrics were green, but provider/tool completion failed within budget.
- Finalization triage points to command discipline / final bearshell evidence and finalization-only continuation, not finish gate weakening.
- `latest` remains `0.3.9-alpha.3` by observation.