# test-fns

![ci_on_commit](https://github.com/ehmpathy/test-fns/workflows/ci_on_commit/badge.svg)
![deploy_on_tag](https://github.com/ehmpathy/test-fns/workflows/deploy_on_tag/badge.svg)

write usecase driven tests systematically for simpler, safer, and more readable code

# purpose

establishes a pattern to write tests for simpler, safer, and more readable code.

by tests defined in terms of usecases (`given`, `when`, `then`) your tests are
- simpler to write
- easier to read
- safer to trust

# install

```sh
npm install --save-dev test-fns
```

# pattern

`given/when/then` is based on [behavior driven design (BDD)](https://en.wikipedia.org/wiki/Behavior-driven_development). it structures tests around:
- **given** a `scene` (the initial state or context)
- **when** an `event` occurs (the action or trigger)
- **then** an `effect` is observed (the expected outcome)

```ts
given('scene', () =>
  when('event', () =>
    then('effect', () => {
      // assertion
    })
  )
);
```

# use

### jest

```ts
import { given, when, then } from 'test-fns';

describe('doesPlantNeedWater', () => {
  given('a dry plant', () => {
    const plant = { id: 7, hydration: 'DRY' };

    when('water needs are checked', () => {
      then('it should return true', () => {
        expect(doesPlantNeedWater(plant)).toEqual(true);
      });
    });
  });
});
```

### vitest

vitest requires a workaround because ESM's thenable protocol prevents direct `then` imports.

the same protocol settles a vitest esm `import { ... } from 'test-fns'` to `given`, `when`, `bdd`, and `getNumberRange` alone — `useThen`, `useBeforeAll`, `useWhen`, `genTempDir`, and the rest import as `undefined`. load them via `createRequire(import.meta.url)('test-fns')` until a fix lands.

**option 1: globals via setup file (recommended)**

```ts
// vitest.config.ts
export default defineConfig({
  test: {
    setupFiles: ['test-fns/vitest.setup'],
  },
});

// your test file - no imports needed
describe('doesPlantNeedWater', () => {
  given('a dry plant', () => {
    const plant = { id: 7, hydration: 'DRY' };

    when('water needs are checked', () => {
      then('it should return true', () => {
        expect(doesPlantNeedWater(plant)).toEqual(true);
      });
    });
  });
});
```

**option 2: bdd namespace**

```ts
import { bdd } from 'test-fns';

describe('doesPlantNeedWater', () => {
  bdd.given('a dry plant', () => {
    const plant = { id: 7, hydration: 'DRY' };

    bdd.when('water needs are checked', () => {
      bdd.then('it should return true', () => {
        expect(doesPlantNeedWater(plant)).toEqual(true);
      });
    });
  });
});
```

### output

both produce:

```sh
 PASS  src/plant.test.ts
  doesPlantNeedWater
    given: a dry plant
      when: water needs are checked
        ✓ then: it should return true (1 ms)
```

# features

### .runIf(condition) && .skipIf(condition)

skip the suite if the condition is not met

```ts
describe('your test', () => {
  given.runIf(onLocalMachine)('some test that should only run locally', () => {
    then.skipIf(onProduction)('some test that should not run against production', () => {
      expect(onProduction).toBeFalse()
    })
  })
})
```

### .repeatably(config)

run a block multiple times to evaluate repeatability. available on `given`, `when`, and `then`.

**`then.repeatably`** — run a test multiple times with environment-aware criteria

```ts
then.repeatably({
  attempts: 3,
  criteria: process.env.CI ? 'SOME' : 'EVERY',
})('it should produce consistent results', ({ attempt }) => {
  const result = generateOutput();
  expect(result).toMatchSnapshot();
});
```

- `attempts`: how many times to run the test
- `criteria`:
  - `'EVERY'`: all attempts must pass (strict, for local development)
  - `'SOME'`: at least one attempt must pass (tolerant, for CI)

**`when.repeatably`** — run the when block multiple times

```ts
given('a probabilistic llm system', () => {
  when.repeatably({
    attempts: 3,
    criteria: 'SOME', // pass if any attempt succeeds
  })('the llm generates a response', ({ attempt }) => {
    then('it should produce valid json', () => {
      const result = generateResponse();
      expect(() => JSON.parse(result)).not.toThrow();
    });
  });
});
```

**`given.repeatably`** — run the given block multiple times

```ts
given.repeatably({
  attempts: 3,
  criteria: 'EVERY', // all attempts must pass (default)
})('different initial states', ({ attempt }) => {
  const state = setupState(attempt);

  when('the system processes the state', () => {
    then('it should handle all variations', () => {
      expect(process(state)).toBeDefined();
    });
  });
});
```

all `repeatably` variants:
- provide an `{ attempt }` parameter (starts at 1) to the callback
- support `criteria: 'EVERY' | 'SOME'` (defaults to `'EVERY'`)
  - `'EVERY'`: all attempts must pass
  - `'SOME'`: at least one attempt must pass (useful for probabilistic tests)

**full block retry with `criteria: 'SOME'`**

when `given.repeatably` or `when.repeatably` uses `criteria: 'SOME'`, the entire block is retried when any `then` block fails:

```ts
when.repeatably({
  attempts: 3,
  criteria: 'SOME',
})('llm generates valid output', ({ attempt }) => {
  // if thenB fails, BOTH thenA and thenB will run again on the next attempt
  then('thenA: output is not empty', () => {
    expect(result.output.length).toBeGreaterThan(0);
  });

  then('thenB: output is valid json', () => {
    expect(() => JSON.parse(result.output)).not.toThrow();
  });
});
```

this enables reliable tests for probabilistic systems where multiple assertions must pass together. if attempt 1 fails on `thenB`, attempt 2 will re-run both `thenA` and `thenB` from scratch.

**what a non-final `'SOME'` attempt absorbs**

under `criteria: 'SOME'`, a failure in any attempt but the last does not fail the file. test-fns withholds it, logs one line per withheld test, and runs the next attempt:

```
🫧  [withheld] attempt 1 failed, a retry follows: <first line of the failure>
```

for a jest snapshot mismatch, the line also names the snapshot (`— Snapshot name: ...`).

when a `useThen` factory or a `useBeforeAll` setup fails, each `then` that reads its result fails too, and each logs its own line (`useThen: tried to access value before test ran`, `usePrep: tried to access value before setup completed`). read the first line of the attempt for the cause; the lines after it are its echo.

this covers every failure kind inside the block:
- a thrown error or a failed `expect`
- a snapshot mismatch
- a timeout — test-fns owns each attempt's deadline, equal to the test timeout you set (via `jest.setTimeout`, `vi.setConfig`, or `testTimeout` in config)
- a `useBeforeAll` / `useBeforeEach` setup that throws or overruns

the final attempt surfaces its own failure, so a block that fails on every attempt still turns the file red. a raw jest/vitest `beforeAll` sits outside the retry and fails the file as before; use `useBeforeAll` for setup the retry should cover.

a timed-out attempt cannot be cancelled — js has no way to stop a promise. its slow call runs on beside the next attempt; the result is discarded, but its side effects (a temp dir, a paid api call) still land. keep a retried body idempotent. on jest, a late `toMatchSnapshot()` mismatch from that slow call can also land in the next attempt's tally, so a clean attempt may be withheld or the file may report an extra failed snapshot — a false red, never a false green.

do not nest a `'SOME'` block inside another `'SOME'` block: the outer block sees no failure from the inner one and stops after its first attempt. the inner block still retries, and its final attempt still turns the file red.

**snapshot keys carry no attempt ordinal**

every attempt checks one baseline. a snapshot inside a repeatable block is keyed on the test name without its `, attempt N` — the reported test name still shows which attempt ran.

> **migration:** a snapshot file written by an earlier version holds keys with `, attempt N`. run the suite once with `-u` (jest) or `--update` (vitest) and commit the snapshot file. until then, a `--ci` run fails with a hint that names the rename:
>
> ```
> New snapshot was not written. The update flag must be explicitly passed to write a new snapshot.
> …
> 🫧  test-fns: this snapshot key was renamed — keys inside a repeatable block no longer carry an attempt suffix such as ", attempt 1". run the suite once with -u, then commit the snapshot file.
> ```

> **vitest `then.repeatably` with `SOME`** now runs the body `attempts` times, as on jest. an earlier version ran it `attempts + 1` times on vitest, so a body that passed only on that extra run now fails; raise `attempts` if it needs the room.

**skip-on-success behavior**

once any attempt passes (all `then` blocks succeed), subsequent attempts are skipped entirely:
- all `then` blocks are skipped
- `useBeforeAll` and `useBeforeEach` setups are skipped
- expensive setup operations do not execute

**recommended pattern for ci/cd**

for reliable ci/cd with probabilistic tests (like llm-powered systems), use environment-aware criteria:

```ts
const criteria = process.env.CI ? 'SOME' : 'EVERY';

when.repeatably({ attempts: 3, criteria })('llm generates response', ({ attempt }) => {
  then('response is valid', () => {
    // strict at devtime (EVERY): all 3 attempts must pass
    // tolerant at cicdtime (SOME): at least 1 attempt must pass
    expect(response).toMatchSnapshot();
  });
});
```

this pattern provides:
- **strict validation at devtime** — `'EVERY'` ensures consistent behavior across all attempts
- **reliable ci/cd pipelines** — `'SOME'` tolerates occasional probabilistic failures while still able to catch systematic issues

## hooks

similar to the sync-render constraints that drove react to leverage hooks, we leverage hooks in tests due to those same constraints. test frameworks collect test definitions synchronously, then execute them later. hooks let immutable references to data be declared before the data is rendered via execution — which enables `const` declarations instead of `let` mutations.

### useBeforeAll

prepare test resources once for all tests in a suite, to optimize setup time for expensive operations

```ts
describe('spaceship refuel system', () => {
  given('a spaceship that needs to refuel', () => {
    const spaceship = useBeforeAll(async () => {
      // runs once before all tests in this suite
      const ship = await prepareExampleSpaceship();
      await ship.dock();
      return ship;
    });

    when('[t0] no changes yet', () => {
      then('it should be docked', async () => {
        expect(spaceship.isDocked).toEqual(true);
      });

      then('it should need fuel', async () => {
        expect(spaceship.fuelLevel).toBeLessThan(spaceship.fuelCapacity);
      });
    });

    when('[t1] it connects to the fuel station', () => {
      const result = useBeforeAll(async () => await spaceship.connectToFuelStation());

      then('it should be connected', async () => {
        expect(result.connected).toEqual(true);
      });

      then('it should calculate required fuel', async () => {
        expect(result.fuelNeeded).toBeGreaterThan(0);
      });
    });
  });
});
```

### useBeforeEach

prepare fresh test resources before each test to ensure test isolation

```ts
describe('spaceship combat system', () => {
  given('a spaceship in battle', () => {
    // runs before each test to ensure a fresh spaceship
    const spaceship = useBeforeEach(async () => {
      const ship = await prepareExampleSpaceship();
      await ship.resetShields();
      return ship;
    });

    when('[t0] no changes yet', () => {
      then('it should have full shields', async () => {
        expect(spaceship.shields).toEqual(100);
      });

      then('it should be ready for combat', async () => {
        expect(spaceship.status).toEqual('READY');
      });
    });

    when('[t1] it takes damage', () => {
      const result = useBeforeEach(async () => await spaceship.takeDamage(25));

      then('it should reduce shield strength', async () => {
        expect(spaceship.shields).toEqual(75);
      });

      then('it should return damage report', async () => {
        expect(result.damageReceived).toEqual(25);
      });
    });
  });
});
```

**when to use each:**
- `useBeforeAll`: use when setup is expensive (database connections, api calls) and tests don't modify the resource
- `useBeforeEach`: use when tests modify the resource and need isolation between runs

### useThen

capture the result of an operation in a `then` block and share it with sibling `then` blocks, without `let` declarations

```ts
describe('invoice system', () => {
  given('a customer with an overdue invoice', () => {
    when('[t1] the invoice is processed', () => {
      const result = useThen('process succeeds', async () => {
        return await processInvoice({ customerId: '123' });
      });

      then('it should mark the invoice as sent', () => {
        expect(result.status).toEqual('sent');
      });

      then('it should calculate the correct total', () => {
        expect(result.total).toEqual(150.00);
      });

      then('it should include the late fee', () => {
        expect(result.lateFee).toEqual(25.00);
      });
    });
  });
});
```

`useThen` creates a test (`then` block) and returns a proxy to the result. the proxy defers access until the test runs, which makes the result available to sibling `then` blocks.

### useWhen

capture the result of an operation at the `given` level and share it with sibling `when` blocks — ideal for idempotency verification

```ts
describe('user registration', () => {
  given('a new user email', () => {
    when('[t0] before any changes', () => {
      then('user does not exist', async () => {
        const user = await findUser({ email: 'test@example.com' });
        expect(user).toBeNull();
      });
    });

    const responseFirst = useWhen('[t1] registration is called', () => {
      const response = useThen('registration succeeds', async () => {
        return await registerUser({ email: 'test@example.com' });
      });

      then('user is created', () => {
        expect(response.status).toEqual('created');
      });

      return response;
    });

    when('[t2] registration is repeated', () => {
      const responseSecond = useThen('registration still succeeds', async () => {
        return await registerUser({ email: 'test@example.com' });
      });

      then('response is idempotent', () => {
        expect(responseSecond.id).toEqual(responseFirst.id);
        expect(responseSecond.status).toEqual(responseFirst.status);
      });
    });
  });
});
```

`useWhen` executes during test collection and returns a value accessible to sibling `when` blocks. use it with `useThen` inside to capture async operation results for cross-block comparisons like idempotency verification.

### how to choose the right hook

| hook            | when to use                                      | execution timing       |
| --------------- | ------------------------------------------------ | ---------------------- |
| `useBeforeAll`  | expensive setup shared across tests              | once before all tests  |
| `useBeforeEach` | setup that needs isolation                       | before each test       |
| `useThen`       | capture async operation result in a test         | during test execution  |
| `useWhen`       | wrap a when block and share result with siblings | during test collection |

**key differences:**
- `useBeforeAll`/`useBeforeEach` - for test fixtures and setup
- `useThen` - for operations that ARE the test (creates a `then` block)
- `useWhen` - wraps a when block at given level, returns result for sibling when blocks (idempotency verification)

### immutability benefits

these hooks enable immutable test code:

```ts
// ❌ mutable - requires let
let result;
beforeAll(async () => {
  result = await fetchData();
});
it('uses result', () => {
  expect(result.value).toBe(1);
});

// ✅ immutable - const only
const result = useBeforeAll(async () => await fetchData());
then('uses result', () => {
  expect(result.value).toBe(1);
});
```

benefits of immutability:
- **safer**: no accidental reassignment or mutation
- **clearer**: data flow is explicit
- **simpler**: no need to track when variables are assigned

## utilities

### genTempDir

generates a temporary test directory within the repo's `.temp` folder, with automatic cleanup of stale directories.

**features:**
- portable across os systems (no os-specific temp dir dependencies)
- timestamp-prefixed names enable age-based cleanup
- slug in directory name helps identify which test created it
- **autoprune** — wire two config keys and a run reclaims its own dirs at teardown
- an age gate backstops it at 24h, for whatever a signal or an unwired run leaves
- optional fixture clone for pre-populated test scenarios

**basic usage:**

```ts
import { genTempDir } from 'test-fns';

describe('file processor', () => {
  given('a test directory', () => {
    const testDir = genTempDir({ slug: 'file-processor' });

    when('files are written', () => {
      then('they exist in the test directory', async () => {
        await fs.writeFile(path.join(testDir, 'example.txt'), 'content');
        expect(await fs.stat(path.join(testDir, 'example.txt'))).toBeDefined();
      });
    });
  });
});
```

**with fixture clone:**

```ts
import { genTempDir } from 'test-fns';

describe('config parser', () => {
  given('a directory with config files', () => {
    const testDir = genTempDir({
      slug: 'config-parser',
      clone: './src/__fixtures__/configs',
    });

    when('config is loaded', () => {
      then('it parses correctly', async () => {
        const config = await loadConfig(testDir);
        expect(config.configOption).toEqual('value');
      });
    });
  });
});
```

**with symlinks to repo root:**

```ts
import { genTempDir } from 'test-fns';

describe('package installer', () => {
  given('a temp directory with symlinks to repo artifacts', () => {
    const testDir = genTempDir({
      slug: 'installer-test',
      symlink: [
        { at: 'node_modules', to: 'node_modules' },
        { at: 'config/tsconfig.json', to: 'tsconfig.json' },
      ],
    });

    when('the installer runs', () => {
      then('it can access linked dependencies', async () => {
        const nodeModules = path.join(testDir, 'node_modules');
        expect(await fs.stat(nodeModules)).toBeDefined();
      });
    });
  });
});
```

symlink options:
- `at` = relative path within the temp dir (where symlink is created)
- `to` = relative path within the repo root (what symlink points to)

notes:
- symlinks are created after clone (if both specified)
- parent directories are created automatically for nested `at` paths
- throws `BadRequestError` if target does not exist
- throws `BadRequestError` if symlink path collides with cloned content

**with git initialization:**

```ts
import { genTempDir } from 'test-fns';

describe('git status helper', () => {
  given('a git repo with committed baseline', () => {
    const testDir = genTempDir({
      slug: 'git-status-test',
      clone: './src/__fixtures__/project',
      git: true,
    });

    when('a file is modified', () => {
      fs.appendFileSync(path.join(testDir, 'config.json'), '\n// comment');

      then('git diff detects the change', () => {
        const diff = execSync('git diff', { cwd: testDir }).toString();
        expect(diff).toContain('+// comment');
      });
    });
  });
});
```

git options:
- `git: true` — init repo, commit 'began', clone/symlink, commit 'fixture'
- `git: { commits: { init: false } }` — init repo only, no commits
- `git: { commits: { fixture: false } }` — commit 'began' only, leave clone/symlink uncommitted

notes:
- repo-local git config is set (ci-safe, no global config needed)
- 'began' commit is empty (created before clone/symlinks)
- 'fixture' commit contains all clone/symlink content
- if no clone/symlink provided, only 'began' commit is created

**directory format:**

```
                    {timestamp}.{run}.{slug}.{8-char-uuid}

# with the autoprune hooks wired — the run slot holds this run's id
.temp/2026-01-19T12-34-56.789Z.r7f3a91c2.my-test.a1b2c3d4/

# with no hooks wired — the run slot holds `_`, which means "no run minted this"
.temp/2026-01-19T12-34-56.789Z._.my-test.a1b2c3d4/
```

the slug helps debuggers identify which test created a directory when they debug.
the run stamp is what lets a run reclaim exactly its own dirs, and no peer's.

**cleanup behavior:**

there are two reclaims, and the first is the one that matters:

1. **the run's own teardown** — reclaims every dir THIS run made, at the end of the run.
   it needs the autoprune hooks (below). it survives a failed test, a `--scope`d run,
   and a peer run beside it.
2. **the age gate** — the backstop. it takes entries older than **24 hours**, and it is
   the only reclaim for a run with no hooks wired, or one killed by a signal.

set `TEST_FNS_MAX_AGE_MS` to widen or narrow the age gate. the `.temp` directory
includes a `readme.md` that restates this policy where a human finds the dirs.

### autoprune

wire these into each runner config. **the guarantee is per runner config**, so a repo with
several configs wires each one.

```ts
// jest.config.ts — TWO keys
export default {
  globalSetup: 'test-fns/autoprune.setup.jest',
  globalTeardown: 'test-fns/autoprune.teardown.jest',
};
```

```ts
// vitest.config.ts — ONE key; the module exports both setup and teardown
export default defineConfig({
  test: { globalSetup: ['test-fns/autoprune.setup.vitest'] },
});
```

with no hooks wired, `genTempDir` behaves exactly as before and prints one notice that
names the config to fix.

**to keep a run's dirs for inspection**, set `TEST_FNS_KEEP`:

```sh
TEST_FNS_KEEP=1 npm test
```

it holds **every** dir the run made — a green run too, since a global teardown is handed
no test results to filter by — and says where they are. the age gate still reclaims them.

### getOneTempDirRoot

yields the temp root, so a consumer derives the path rather than hardcodes it.

```ts
import { getOneTempDirRoot } from 'test-fns';

const { pathPhysical, pathSymlink } = getOneTempDirRoot();
// pathPhysical => /tmp/test-fns/{repo}/.temp
// pathSymlink  => {gitRoot}/.temp/genTempDir.symlink
```

use it to count what a run left behind. **filter through `isTempDir`** — the root also
holds a `readme.md` and a `.gitignore`, so a raw entry count reads 2 on a pristine root.

### isTempDir

checks if a path is a test directory created by genTempDir.

```ts
import { isTempDir } from 'test-fns';

isTempDir({ path: '/repo/.temp/2026-01-19T12-34-56.789Z.my-test.a1b2c3d4' }); // true
isTempDir({ path: '/tmp/random' }); // false
```

## slowtest reporter

identify slow tests in your test suite with hierarchical time visibility.

### what it does

the slowtest reporter runs after your tests complete and shows:
- which test files are slow (above a configurable threshold)
- nested hierarchy breakdown with time spent in each `given`/`when`/`then` block
- hook time (setup/teardown) separated from test execution time

### jest configuration

```ts
// jest.config.ts
import type { Config } from 'jest';

const config: Config = {
  reporters: [
    'default',
    ['test-fns/slowtest.reporter.jest', {
      slow: '3s',                        // threshold (default: 3s for unit, 10s for integration)
      output: '.slowtest/report.json',   // optional: export json report
      top: 10,                           // optional: limit terminal output to top N slow files
    }],
  ],
};

export default config;
```

### vitest configuration

```ts
// vitest.config.ts
import { defineConfig } from 'vitest/config';
import SlowtestReporter from 'test-fns/slowtest.reporter.vitest';

export default defineConfig({
  test: {
    reporters: [
      'default',
      new SlowtestReporter({
        slow: '3s',
        output: '.slowtest/report.json',
      }),
    ],
  },
});
```

### terminal output

after tests complete, you'll see a report like:

```
slowtest report:
----------------------------------------------------------------------
🐌 src/invoice/invoice.test.ts                              8s 710ms [SLOW]
   └── given: [case1] overdue invoice                       7s 230ms
       ├── (hooks: 340ms)
       └── when: [t0] nurture triggered                     6s 890ms
           ├── then: sends reminder email                   6s 10ms
           └── then: logs notification                      880ms

🐌 src/auth/login.test.ts                                   3s 500ms [SLOW]

----------------------------------------------------------------------
total: 15s 10ms
files: 4
slow: 2 file(s) above threshold
```

the report shows:
- 🐌 emoji marks slow files
- hierarchical breakdown with `given` > `when` > `then` structure
- hook time displayed as `(hooks: Xms)` when setup contributes to block duration
- summary with total time, file count, and slow file count

### configuration options

| option | type | default | description |
|--------|------|---------|-------------|
| `slow` | `number \| string` | `3000` (3s) | threshold in ms or human-readable string (`'3s'`, `'500ms'`) |
| `output` | `string` | — | path to write json report (e.g., `.slowtest/report.json`) |
| `top` | `number` | — | limit terminal output to top N slowest files |

### json output format

when `output` is configured, the reporter writes a json file:

```json
{
  "generated": "2026-01-31T14:23:00Z",
  "summary": {
    "total": 15010,
    "files": 4,
    "slow": 2
  },
  "files": [
    {
      "path": "src/invoice/invoice.test.ts",
      "duration": 8710,
      "slow": true,
      "blocks": [
        {
          "type": "given",
          "name": "[case1] overdue invoice",
          "duration": 7230,
          "hookDuration": 340,
          "blocks": [
            {
              "type": "when",
              "name": "[t0] nurture triggered",
              "duration": 6890,
              "hookDuration": 0,
              "tests": [
                { "name": "then: sends reminder email", "duration": 6010 },
                { "name": "then: logs notification", "duration": 880 }
              ]
            }
          ]
        }
      ]
    }
  ]
}
```

use the json output for:
- trend analysis over time
- ci shard optimization (distribute tests by time)
- integration with other tools