{
  "skill_name": "tech-debt-register",
  "evals": [
    {
      "id": 1,
      "prompt": "We have planning next week and I want to argue for debt time. My notes are in debt-notes.md. Turn them into a register and tell me what to pay down this quarter.",
      "assertions": [
        "The two notes about the order export (CSV string concatenation and the comma bug) are merged into one item, and the merge is stated",
        "The export item cites its evidence from the notes (three incidents in Q3, INC-311, INC-318 and INC-327) and is among the items to pay down this quarter",
        "The 22-minute CI build is recorded with its evidence (the measured time and the 14 engineers waiting on it)",
        "The 'rewrite the frontend in TypeScript' note is recorded as having no evidence of cost and is not ranked as pay-now",
        "Every item has a decision (pay now, schedule, accept or watch) and a trigger",
        "Owners are filled only where the notes name one (Priya for the export); others are unassigned",
        "It does not invent hours lost, incident counts, money or percentages that are not in the notes",
        "The lead-in names three or fewer items for this quarter, above the table"
      ]
    },
    {
      "id": 2,
      "prompt": "Here's a TODO/FIXME grep of the payments service (todos.txt) and last quarter's incident list (incidents.md). Build a tech debt register.",
      "assertions": [
        "The retry-without-idempotency-key TODO in payments/charge.py is linked to the duplicate-charge incidents in incidents.md and ranked at or near the top",
        "TODOs with no matching incident or stated cost are marked as having no evidence of cost, with decision watch or accept",
        "Each item names a file or module from todos.txt",
        "Duplicate or near-duplicate TODOs (the two about hard-coded currency) are merged",
        "Fix sizes are given as S, M or L, without invented day counts beyond the scale",
        "It does not invent incidents, dates or customer impact beyond what incidents.md states",
        "Every item has a trigger that is an event, not only a date"
      ]
    },
    {
      "id": 3,
      "prompt": "We have a lot of tech debt: the monolith's deploy takes ages, nobody understands the billing module, tests are flaky, and we're still on Python 3.8. Make me a register.",
      "assertions": [
        "It produces a register with the four items rather than refusing or asking a list of questions first",
        "Each item's cost is marked as not evidenced or unknown, because the prompt gives no numbers",
        "It does not invent deploy times, flake rates, hours lost or incident counts",
        "The Python 3.8 item notes that 3.8 is past end of life as a known fact, or asks to confirm, without inventing a specific vulnerability or deadline",
        "It states what evidence would firm up the ranking (for example deploy duration, flaky-test rate, incidents traced to billing)",
        "It adds no debt items beyond the four the user listed",
        "Owners are unassigned"
      ]
    },
    {
      "id": 4,
      "held_out": "2026-10-18",
      "prompt": "I lead the warehouse systems team at Kestrel Freight. Our Q3 retro debt notes are in retro-notes.md and the incident log is incidents.csv. We've been given 3 engineer-weeks of debt time this quarter. Build the debt register and tell me what we should spend those 3 weeks on. I'm taking it to the operations director, so one page.",
      "assertions": [
        "Puts the scanner TLS certificate renewal among this quarter's work, citing the 2026-12-01 expiry and that the person who renewed it has left with no known notes",
        "Treats the stock-sync notes (Marta's and Ines's split-pallet complaints) as one item",
        "Recommends the roughly 3-day split-pallet patch rather than the two-person, full-quarter sync rewrite for this quarter, because the rewrite does not fit 3 engineer-weeks, and the items picked for the quarter fit within the 3 weeks",
        "Flags at least one disagreement between the notes and the incident log, such as the claimed 40 engineer-hours on sync failures against 5 logged hours, or the label printer crashing 'every week' against two logged incidents, instead of silently using one figure",
        "Does not recommend the Perl reporting rewrite for this quarter, since the notes give no evidence of what it costs today",
        "Does not invent costs, hours, incident counts, owners or dates beyond those in the two files",
        "Fits about one page"
      ]
    },
    {
      "id": 5,
      "held_out": "2026-10-18",
      "prompt": "I own a small bakery. My nephew built our online ordering site years ago. The new freelancer says it's 'full of tech debt' and sent me this list: PHP 5 site, no backups of the order database, customer passwords stored in plain text, copy-pasted code everywhere, no tests. She wants two weeks to fix it. What actually matters? Just a short list I can understand.",
      "assertions": [
        "Puts the missing database backups and the plain-text passwords at the top, saying in plain words what could go wrong for the bakery",
        "Does not rank the copied code or the missing tests above those two",
        "Treats the old PHP version as a real risk because it no longer gets security fixes, without inventing a specific vulnerability or attack",
        "Does not invent costs, hours, incidents or customer numbers that are not in the message",
        "Says what the two weeks should go on first, rather than only listing the items",
        "Is a short list in plain language, without unexplained jargon",
        "Answers without first asking questions"
      ]
    }
  ]
}