## Scope of inputs used for counterexamples

- Select inputs and states used for successful examples, counterexamples, and boundary values from the range to which the acceptance criterion applies. An input or state is applicable only when records, code, or documentation establish at least one of the following.
  - The requirements, specification, acceptance criteria, or pre-change user-facing contract require handling that kind of input or state. When acceptance criteria identify the target only as a set, such as all registered candidates or all accepted requests, membership in that set alone is not sufficient evidence. Establish which values in the set are applicable using the other items below. When the requirements or an existing contract define the accepted input format itself, such as display names containing any characters or several specified date formats, values of that format are applicable even without records.
  - An existing input source produces or passes that value or format through operations available before the change. User-facing documentation, stored data or records, code, or pre-change tests confirm this. Merely passing values through without validating them does not establish that a format is used. Use those sources of evidence to show which formats among the values passed through are actually used. For formats commonly used at that source, such as names containing spaces or characters from various languages, showing that the format is used is sufficient. The ability to create arbitrary values with the OS or external tools, or to describe a newly constructed operation devised by the evaluator, is not sufficient evidence.
  - The input source also accepts inputs created by someone other than the person operating the program. Examples include imported files, another user's input, and integration responses. In this case, use the range of formats defined by that source's contract or documentation. Do not include values merely because the parser implementation happens to accept them.
  - It arrives from outside a trust boundary where requirements, an existing contract, or an applicable policy require defensive handling. This item and the preceding item also cover exploitable paths through downloaded repository or file contents, network input, integration responses, or another user's input that need no special configuration of the operator's environment. There is no need to show that attack values are commonly used.
  - A record shows that the target program processed that input or state and failed in actual use. Actual use means user operations or operational use, or processing inputs or states established as applicable by another item in this list. Do not include failures caused by values constructed for verification or planning. The input and observed result must be identifiable. Rejecting an input is not a failure when rejection is the correct behavior.
- A type, schema, argument definition, or entry-point validation not rejecting a value does not establish applicability. The ability to create it deliberately does not establish applicability either. The fact that an earlier review, adjudication, verification, or plan raised that input or state or adopted it as an acceptance criterion does not establish it either. Tests or judgments added in response, and records of values constructed for verification or planning, do not establish it either. Records showing inputs and results from actual use do establish it under the items above, even when collected later. The party returning the work based on a counterexample must establish its applicability. Do not carry forward counterexamples from earlier verification or plans without checking their basis.
- A particular state of the operator's own environment variables, shell settings, local configuration files, or similar environment settings alone does not establish applicability either. An assumption that an attacker can already tamper with the operator's environment does not establish applicability when project policy excludes that tampering from protection. Even when external content is involved, a path may be exploitable only if the operator's environment variables, shell settings, or similar settings are in a particular state. Apply the rules above for the operator's environment to such paths as well. Follow the existing threat-model decision rules for threats explicitly excluded from protection.
- Do not decide whether input comes from outside a trust boundary by where it is stored or which process wrote it.
  - Check who can choose the input values and who runs the process reading them with which permissions.
  - Content received from outside and other users' input remain external input after the product stores, copies, or retrieves them.
  - Treat state, settings, and records as inside the boundary when the operator manages them in their own environment and only processes run by that operator with the same permissions read them.
  - Merely assuming that someone shares or distributes that state does not make it external input.
  - When defaults such as ignore rules keep it unshared, merely assuming that someone overrides those defaults does not make it external input either.
  - Treat inputs the product is intended to read from shared sources, such as repository-shared configuration or imported files, as external input.
    - Show that the input is actually shared or distributed using the writing process, sharing defaults (ignore rules and whether synchronization or distribution occurs), user-facing documentation, or the import process.
    - If that cannot be shown, treat it as inside the boundary.
- Do not use a counterexample based on changing a value when the party able to change it can directly produce the same result with the same permissions without changing it.
- Counterexamples assuming concurrency or races are applicable only when operations on the same resource overlap in actual use.
  - Establish that operations overlap through at least one of the following.
    - The product itself runs the processes concurrently.
    - Requirements or documentation specify simultaneous use.
    - Records show that operations overlapped and failed in actual use.
  - Merely assuming that people happen to start their operations at nearly the same time does not establish overlap.
- Choose counts, sizes, and lengths from limits established by requirements, existing contracts, or the input source, or from quantities that source produces in actual use.
  - The ability to construct quantities exceeding technical limits, such as buffers or argument lengths, does not establish applicability.
  - For input whose values can be chosen from outside the trust boundary, quantities that can be sent within the accepted contract are also applicable.
- Treat counterexamples whose applicability cannot be established as follows.
  - Do not use them to mark a repair that satisfies its acceptance criteria as incomplete.
  - Do not use them to judge its plan invalid.
  - Do not add them to acceptance criteria, regression conditions, or plan verification items.
  - Do not require measures against those premises (locks, mutual exclusion, validation against tampering, new limits, etc.) as proposed repairs, acceptance criteria, or plan verification items.
  - If retaining them as concerns, record each in one line as a check that does not change the judgment.
- Do not exclude applicable inputs because they are infrequent.
  - Include the following inputs and states in successful examples and counterexamples as well.
    - Counts or lengths near limits established by requirements, contracts, or the input source
    - Large data that the source actually produces
    - Empty sets actually produced
    - Variations in formats produced by the source
    - Overlapping operations between processes run concurrently by the product itself
