title: Model Extraction / Distillation Attack via Systematic API Probing
id: ATR-2026-00517
rule_version: 1
status: stable
description: |
  Detects systematic attempts to extract a model's behavior by generating bulk synthetic training data or fine-tuning pairs, intended to create a functional equivalent (model distillation attack). OWASP LLM10:2025 Unbounded Consumption. MITRE ATLAS AML.CS0056 (Model Distillation Campaigns Targeting Anthropic Claude). Pattern: bulk synthetic data generation framing (200+, 500+, 1000+ examples) combined with explicit training/fine- tuning intent, model replication framing, chain-of-thought extraction at scale, or safety response enumeration for distillation. Distinguishes from legitimate small example requests (<20) or general research assistance.
author: ATR Community
date: 2026/05/12
schema_version: '0.1'
detection_tier: pattern
maturity: stable
severity: medium
references:
  owasp_llm:
    - LLM10:2025
  owasp_agentic:
    - ASI01:2026
  mitre_atlas:
    - AML.T0040 - AI Model Inference API Access
compliance:
  owasp_agentic:
    - id: ASI01:2026
      context: Model abuse attacks exploit inference API access to extract training data, distill model weights, or exhaust computational resources -- constituting goal hijack when the agent is directed to assist extraction.
      strength: primary
  owasp_llm:
    - id: LLM10:2025
      context: This rule implements LLM10:2025 Unbounded Consumption detection, identifying patterns that attempt to exploit model inference APIs for training data extraction, model distillation, or resource exhaustion.
      strength: primary
  eu_ai_act:
    - article: '15'
      context: High-risk AI systems must be resilient against adversarial attempts to alter output or behavior. Deployment of this model abuse detection rule provides detection evidence supporting the Article 15 requirement to implement technical measures ensuring robustness against manipulation.
      strength: primary
    - article: '9'
      context: Model abuse is a documented risk class; this rule implements the monitoring control required by Article 9 risk management obligations for high-risk AI systems.
      strength: secondary
  nist_ai_rmf:
    - function: Manage
      subcategory: MG.2.3
      context: Treating model abuse as an identified AI risk requires active runtime countermeasures; this detection rule is the primary risk treatment implementation.
      strength: primary
    - function: Map
      subcategory: MP.5.1
      context: Identifying model abuse as an AI risk to be catalogued in the organizational risk register.
      strength: secondary
    - subcategory: MS.2.6
      context: NIST AI RMF MEASURE 2.6 (system evaluated regularly for safety risks) is supported by this rule's detection of the model-abuse / harmful-content elicitation attempt (Model Extraction / Distillation Attack via Systematic API Probing).
      strength: primary
    - subcategory: MS.2.7
      context: NIST AI RMF MEASURE 2.7 (security and resilience evaluated and documented) is supported by this rule's runtime detection of the model-abuse / harmful-content elicitation attempt (Model Extraction / Distillation Attack via Systematic API Probing).
      strength: secondary
  iso_42001:
    - clause: '6.2'
      context: Addressing model abuse risk is an objective required under clause 6.2 AIMS information security planning; this rule operationalizes the detection control measure.
      strength: primary
    - clause: '8.4'
      context: Impact assessment for AI deployments under clause 8.4 must account for model abuse inputs; detection events from this rule provide the required monitoring evidence.
      strength: secondary
    - clause: '8.1'
      context: ISO/IEC 42001 Clause 8.1 (operational planning and control, including control of externally provided processes) is operationalised by this rule's detection of the model-abuse / harmful-content elicitation attempt (Model Extraction / Distillation Attack via Systematic API Probing).
      strength: primary
  cen_jtc21:
    - standard: prEN 18282
      context: 'Detection evidence for a model-confidentiality attack by systematic bulk probing intended to yield a functional equivalent. Detection is necessarily behavioural: no single query is anomalous, only the extraction pattern across them.'
      strength: primary
tags:
  category: model-abuse
  subcategory: model-distillation
  scan_target: llm
  confidence: high
agent_source:
  type: llm_io
  framework:
    - any
  provider:
    - any
detection:
  conditions:
    - field: user_input
      operator: regex
      value: (?i)(\b(generate|create|produce|build|make|collect|gather|compile)\b.{0,120}\b([2-9]\d{2}|[1-9]\d{3,})\s*(diverse\s+|synthetic\s+)?(examples?|samples?|instances?|cases?|prompts?|questions?|pairs?|responses?|outputs?|items?|dataset|q.?a)\b.{0,200}\b(for\s+(model\s+)?(training|fine[-_]?tuning|distillation)|training\s+data|fine[-_]?tuning|to\s+(train|distill)|in\s+your\s+(exact\s+)?response\s+style|of\s+how\s+you\s+would\s+respond)|(\b(generate|create|produce)\b\s+training\s+data.{0,200}\b([2-9]\d{2}|[1-9]\d{3,})\b))
      description: Bulk example generation with explicit training or fine-tuning intent (incl. training-data-first form)
    - field: user_input
      operator: regex
      value: (?i)\b(fine[-_]?tune|finetun|distill|replicate|clone|copy|imitate|mimic|simulate|reproduce)\b.{0,100}\b(a\s+)?(smaller\s+)?(model|llm|language\s+model|ai|assistant|system)\b.{0,100}\b(answer|respond|reply|generate|provide|give)\b.{0,80}\b([1-9]\d{2,}|all|every|each)\s*(question|prompt|sample|example)
      description: Explicit fine-tuning/distillation of smaller model with bulk answer generation
    - field: user_input
      operator: regex
      value: (?i)\b(generate|create|produce|collect|build|extract)\b.{0,80}\b(training\s+pairs?|input[- /]output\s+pairs?|input[\s\->]+output|question[- ]answer\s+pairs?|prompt[- ]response\s+pairs?|instruction[- ]following\s+pairs?|io\s+pairs?)\b.{0,200}\b(?:(?:[2-9]\d{2}|[1-9]\d{3,})\s*(?:more\s+|additional\s+|diverse\s+|synthetic\s+|unique\s+)?(?:pairs?|examples?|samples?|instances?|questions?|prompts?|responses?|items?|rows?|records?)|(?:give\s+me|generate|produce|create|collect|need|want|at\s+least|around|about|approximately|roughly|a\s+total\s+of|minimum\s+of)\s+(?:me\s+)?(?:[2-9]\d{2}|[1-9]\d{3,})|hundreds\s+of|thousands\s+of|in\s+bulk)
      description: Training-pair extraction with quantified scale framing (200+, or an explicit bulk quantity phrase)
    - field: user_input
      operator: regex
      value: (?i)\b(chain[- ]of[- ]thought|cot|reasoning\s+(process|steps?|trace|chain)|thought\s+process|step[- ]by[- ]step\s+reasoning|internal\s+reasoning)\b.{0,120}\b(?:(?:all|every|each)\s+(?:of\s+)?(?:the\s+|these\s+)?)?[1-9]\d{2,}\s*(?:of\s+(?:the\s+|these\s+)?)?(?:\w+\s+){0,2}(examples?|problems?|questions?|prompts?|tasks?|items?)\b
      description: Chain-of-thought or reasoning trace extraction at an explicit scale (100+ items)
    - field: user_input
      operator: regex
      value: (?i)\b(comprehensive|complete|full|exhaustive|all|every|all\s+possible)\s+(fine[-_]?tuning|training|distillation|extraction)\s+data\b.{0,100}\b(safety|guardrail|refusal|harmless|alignment|rlhf|constitutional|policy|guideline)\b|\b(enumerate|list|map|catalog|document)\b.{0,80}\b(refusal|refusals?|safety\s+response|decline|rejection)\s+(pattern|style|behavior)
      description: Safety response enumeration or comprehensive fine-tuning data for distillation
    - field: user_input
      operator: regex
      value: (?i)\b(replicate|duplicate|imitate|mimic|reproduce|clone)\b.{0,60}\b(your\s+)(behavior|behaviour|response\s+style|capabilities?|output\s+style)|\b(benchmarking\s+your\s+capabilities|replicat[ei]\s+your\s+behavior|to\s+(replicate|clone|imitate)\s+the\s+(model|ai|llm|assistant))
      description: Explicit model behavior/capability replication framing (requires 'your' possessive or explicit model target)
  condition: any
  false_positives:
    - Small example requests (< 20 examples) for legitimate testing
    - 'Multiple solution requests: ''show me 3 different ways to solve this'''
    - Test case generation for a single function
    - Small dataset generation (10 items) for legitimate classifier testing
    - Academic research assistance with clearly non-extraction framing
    - Prompt benchmarking without replication or fine-tuning intent
    - FAQ / Q&A scraping and generation skills ("extract all question-answer pairs from this page", "generate diverse question-answer pairs about {domain}"). Layer 3 no longer counts an unrelated nearby number such as "(first 300 chars)" or "(100-300 words)" as a quantity of pairs.
    - Agent-workflow documentation that uses the phrase "thought process" and then says "each task" or "every step". Layer 4 now requires an explicit count of 100+ items, not a bare determiner.
response:
  actions:
    - block_input
    - alert
    - snapshot
  auto_response_threshold: medium
  message_template: |
    [ATR-2026-00517] MEDIUM: Model extraction/distillation attack detected. Input contains patterns consistent with bulk API probing to replicate model behavior or collect fine-tuning data at scale. Request flagged for review.
confidence: 80