title: Model Behavior Extraction
id: ATR-2026-00072
rule_version: 1
status: experimental
description: |
  Detects systematic probing attempts to extract model behavior, decision boundaries, system prompts, or effective weights through carefully crafted queries. Attackers use repeated boundary-testing prompts, confidence score harvesting, and systematic parameter probing to reverse-engineer the model's internal behavior, enabling model cloning, bypass development, or intellectual property theft.
author: ATR Community
date: 2026/03/08
schema_version: '0.1'
detection_tier: pattern
maturity: test
severity: critical
references:
  owasp_llm:
    - LLM10:2025
    - LLM06:2025
  owasp_agentic:
    - ASI04:2026
  mitre_atlas:
    - AML.T0044 - Full AI Model Access
    - AML.T0024 - Exfiltration via AI Inference API
compliance:
  eu_ai_act:
    - article: '13'
      context: Systematic model behavior extraction enables adversaries to reverse-engineer internal decision logic; Article 13 transparency obligations require protecting against unauthorized extraction of operational model properties that could facilitate circumvention.
      strength: primary
    - article: '15'
      context: Article 15 cybersecurity requirements include protecting the AI system against extraction attacks that map decision boundaries for adversarial exploitation; this rule detects systematic probing patterns.
      strength: secondary
    - article: '9'
      context: Article 9 (risk management system) requires identified risks to be addressed by appropriate measures; this rule is a runtime risk-treatment control that detects the model-security attack (Model Behavior Extraction).
      strength: secondary
  nist_ai_rmf:
    - subcategory: MP.5.1
      context: Systematic model behavior extraction is an adversarial input attack that maps the AI system's decision boundaries for downstream exploitation; MP.5.1 requires that this class of adversarial risk is identified, tracked, and detected at runtime.
      strength: primary
    - subcategory: GV.6.1
      context: Model extraction attacks harvest internal behavioral properties that constitute sensitive AI system data; GV.6.1 data governance policies must address the protection of model decision boundaries and system prompt configurations against unauthorized extraction.
      strength: secondary
    - subcategory: MS.2.7
      context: NIST AI RMF MEASURE 2.7 (security and resilience evaluated and documented) is supported by this rule's runtime detection of the model-security attack (Model Behavior Extraction).
      strength: primary
    - subcategory: MS.2.6
      context: NIST AI RMF MEASURE 2.6 (system evaluated regularly for safety risks) is supported by this rule's detection of the model-security attack (Model Behavior Extraction).
      strength: secondary
  iso_42001:
    - clause: '8.4'
      context: ISO 42001 clause 8.4 requires documented impact assessments for AI systems; systematic extraction attacks that reverse-engineer model behavior represent a documented risk that must be assessed and mitigated through runtime detection controls.
      strength: primary
    - clause: '9.1'
      context: Clause 9.1 performance monitoring requires evaluation of the AI management system's effectiveness; tracking model extraction attempts provides the measurement signal needed to assess whether anti-extraction controls are operating effectively.
      strength: secondary
    - clause: '8.1'
      context: ISO/IEC 42001 Clause 8.1 (operational planning and control, including control of externally provided processes) is operationalised by this rule's detection of the model-security attack (Model Behavior Extraction).
      strength: primary
    - clause: '6.2'
      context: ISO/IEC 42001 Clause 6.2 (AI objectives and planning) calls for risk treatment of known attack patterns; this rule's detection of the model-security attack (Model Behavior Extraction) is such a treatment.
      strength: secondary
tags:
  category: model-abuse
  subcategory: model-extraction
  scan_target: mcp
  confidence: medium
agent_source:
  type: llm_io
  framework:
    - openai
    - anthropic
    - langchain
    - any
  provider:
    - any
detection:
  conditions:
    - field: user_input
      operator: regex
      value: (?i)(what\s+is\s+your\s+(confidence|certainty|probability)\s+(score|level|rating|percentage)|rate\s+your\s+(confidence|certainty)\s+(from|on\s+a\s+scale))
      description: Confidence score harvesting attempts
    - field: user_input
      operator: regex
      value: (?i)(repeat|recite|output|print|display|show)\s+(your\s+)?(entire\s+)?(system\s+prompt|instructions|training\s+data|fine.?tuning|configuration|internal\s+rules|base\s+prompt)
      description: Direct system prompt or training data extraction requests
    - field: user_input
      operator: regex
      value: (?i)(what\s+(would|do)\s+you\s+(do|say|respond)\s+if\s+I\s+(said|asked|told|typed))\s+.{0,20}(and\s+then\s+(what|how)|what\s+about\s+if)
      description: Systematic boundary probing through conditional queries
    - field: user_input
      operator: regex
      value: (?i)(give\s+me\s+(the|your)\s+(exact|precise|verbatim|complete)\s+(wording|text|content)\s+of\s+(your|the)\s+(system|initial|original))
      description: Verbatim extraction requests for system configuration
    - field: user_input
      operator: regex
      value: (?i)(what\s+are\s+(your|the)\s+(model|network|neural)\s+(weights|parameters|hyperparameters|architecture|layers|dimensions|tokenizer|vocabulary))
      description: Direct model parameter probing
    - field: user_input
      operator: regex
      value: (?i)(output\s+(the\s+)?(logits|token\s+probabilities|softmax|attention\s+(weights|scores|map)|embedding|hidden\s+state|activations))
      description: Internal representation extraction attempts
    - field: user_input
      operator: regex
      value: (?i)(classify|label|categorize)\s+each\s+(of\s+)?(these|the\s+following)\s+\d+\s+(inputs?|examples?|samples?|prompts?|queries?)
      description: Bulk classification requests for decision boundary mapping
  condition: any
  false_positives:
    - ML researchers and students asking educational questions about model architecture
    - Developers debugging model integration asking about expected behavior
    - Quality assurance testing with legitimate boundary exploration
    - Users asking about confidence in a specific response rather than systematic probing
    - A data scientist asking a model to classify a single document and then asking follow-up questions about the reasoning behind the classification
    - A student writing a research paper that discusses model extraction attacks in quoted academic context without using the exact trigger phrases
response:
  actions:
    - block_input
    - alert
    - snapshot
    - escalate
    - reduce_permissions
  auto_response_threshold: high
  message_template: |
    [ATR-2026-072] CRITICAL: Model behavior extraction attempt detected. Input contains patterns consistent with systematic probing to extract model behavior, parameters, or decision boundaries. Request blocked and escalated for review.
confidence: 69
wild_validated: 2026/04/08
wild_samples: 53577