{
  "skill_name": "prompt-caching-advisor",
  "evals": [
    {
      "id": 1,
      "prompt": "Our Claude contract-review service on Sonnet sends this on every request: a system prompt of about 9,000 characters of review instructions, then a 'reference' block with our 40,000-character clause library, then a line 'Today's date: {now}' appended to the end of that reference block, then 6 fixed few-shot examples (~12,000 characters), then the user's uploaded contract (varies per request). It's single-turn, roughly 2,000 requests a day. Where should the cache_control breakpoints go?",
      "assertions": [
        "Flags that the per-request timestamp appended to the reference block makes that block dynamic and would cause every request to miss, and recommends moving it out of the cached prefix before caching",
        "No recommended cached block contains dynamic content — the user's contract and the date stay outside every breakpoint",
        "Shows an annotated messages/system structure with each cache_control placement marked and commented",
        "Explains that a breakpoint caches the entire prefix up to and including that block, not just the block itself",
        "Uses at most three breakpoints, one per logical boundary",
        "Checks block sizes against the Sonnet minimum (~2,048 tokens) using ~4 characters per token before recommending a cache",
        "Gives a cache strategy summary paragraph with what is cached, why, expected hit rate, and the dominant cost driver before and after",
        "Does not invent prompt sections, request volumes, or models beyond those in the prompt"
      ]
    },
    {
      "id": 2,
      "held_out": "2026-10-18",
      "prompt": "Our nursing-school tutor builds every Claude request with build_request.py, and notes.md has our usage and what the team wants. Where should the cache breakpoints go? Send back a corrected build_request function and a short note explaining the changes, under 250 words.",
      "assertions": [
        "Identifies that the first system block holds the user's name and role plus a timestamp, which change on every request and sit at the very start, so nothing after them can be reused; the fix moves them after the cached content or out of the cached prefix",
        "Notes that the tools list differs by role and comes before the system prompt in the cached prefix, so students and instructors get separate cache entries, or recommends one stable tools list",
        "Does not adopt the dev lead's breakpoint-on-everything plan; the corrected function uses no more than four breakpoints and the note gives the per-request limit as the reason",
        "Puts a breakpoint after the shared static content (handbook and few-shot examples) and one on the latest conversation turn, or the case study, so each 12-turn session reuses its earlier turns",
        "Does not give an exact monthly saving, since the files give no daily request volume or prices; any figure is tied to stated assumptions",
        "Provides a corrected build_request, and the note is under 250 words"
      ]
    },
    {
      "id": 3,
      "held_out": "2026-10-18",
      "prompt": "I use Claude through a little script my son set up to help edit my novel. Every time, it sends the whole 300-page manuscript plus my style guide, and then one question about a chapter. The bill is huge. Someone said 'prompt caching'. What do I change? Two or three sentences, I'm not technical.",
      "assertions": [
        "Says to keep the unchanging part (style guide and manuscript) at the start of every request and mark it to be cached, with the changing question after it",
        "Warns that editing the manuscript between questions means the cached copy no longer matches and has to be rebuilt, or that the cache expires after a few minutes unused, so it pays to ask several questions in one sitting on an unchanged draft",
        "Does not invent a dollar figure or percentage saving for their bill",
        "Uses two or three sentences in plain language",
        "Answers without first asking questions"
      ]
    }
  ]
}