Evaluation Science Brief
What Does an Agent Evaluation Actually Support?
A benchmark result is an observation. A deployment decision is an action. Several inferences connect the two. For example:
System A achieved 74% on benchmark B under configuration C.
This may accurately describe the evaluation result. It does not establish that System A will achieve similar performance across the broader task domain, perform similarly in workflow X, or meet the requirements for use there. Those conclusions require different evidence. The evaluation must score the intended outcome correctly. The sampled tasks and runs must represent the evaluation domain. The evaluation setting must correspond sufficiently to the target workflow. The measured performance must then satisfy the criteria used for the decision. Argument-based validity separates these as scoring, generalization, extrapolation, and decision inferences [1,2].
Observed runs
│ scoring
▼
Evaluation result
│ generalization
▼
Performance in the evaluation domain
│ extrapolation
▼
Expected performance in the target setting
│ decision
▼
Supported action or conclusion
Scoring
The scoring inference asks whether the evaluation mechanism correctly identified success, failure, or quality in the observed artifact, trajectory, or environment state. Tests may check only part of the intended state. A parser may inspect the wrong artifact. An agent may reach the rewarded state through an unintended shortcut [3,10]. Evidence for this step comes from validating the grader and evaluation protocol.
Generalization
The generalization inference asks whether the evaluated tasks, runs, and conditions support an estimate for the intended evaluation domain. Task sampling, stochastic execution, prompt or scaffold choices, and other execution conditions can all contribute variation [4–6]. This step requires a defined task frame and an uncertainty analysis that reflects the important sources of variation.
Extrapolation
The extrapolation inference asks whether performance in the evaluation domain is informative about the target setting. The relevant comparison can include task distribution, users, tools, permissions, system configuration, workflow state, resource constraints, oversight, and consequences of failure [7,8]. A benchmark name does not establish these correspondences. They have to be examined directly.
Decision
The decision inference asks whether performance in the target setting satisfies the criteria for the action being considered. Failure costs, alternatives, latency, resource use, safety requirements, acceptable error thresholds, and the distribution of failures can all affect the decision [9]. A performance estimate therefore does not itself determine whether the system should be used.
Record the system and evaluation separately
For agents, the model name alone is usually incomplete. The system identity should record the model version, scaffold, prompts or policies, tools, permissions, and relevant versions or date. The evaluation identity should separately record the benchmark or task-set version, environment, grader, tests, parser, run budget, retry policy, and other conditions that determine how behavior becomes a result. This makes it possible to distinguish a change in the evaluated system from a change in how it was evaluated.
State the task frame and run policy
A domain-level estimate requires a defined target task population, evaluation task frame, and task-selection process. The run policy should also state how timeouts, budget limits, tool failures, harness crashes, human interventions, unanswered clarification requests, and partial completion are handled. Different policies can produce different estimates from the same underlying runs.
Preserve trajectory and reliability evidence when it matters
Agents can act on the environment used to evaluate them. They may exploit evaluation artifacts, infer task structure, use unintended feedback channels, or reach a rewarded state without performing the intended task [10]. When this is plausible, trajectory-level evidence may be needed to establish what actually happened. Aggregation can also hide operational differences. Two agents can have the same mean while one fails consistently on a known subset of tasks and another fails unpredictably across repeated attempts. If those differences matter downstream, the evaluation record should preserve them.
Where the evidence stops
An evaluation should state the strongest conclusion supported by the available evidence. For example:
Decision being considered:
Use A for workflow X
Observed runs ✓
Scoring ✓
Evaluation-domain generalization ✓
Target-workflow correspondence ?
Decision evidence ✕
SUPPORTED CONCLUSION:
A's performance has been established in the defined
evaluation domain under the evaluated configuration.
ADDITIONAL EVIDENCE NEEDED:
Evaluate correspondence with workflow X and specify
the decision threshold and failure criteria.This identifies the first unsupported step and the study needed to address it.
Evaluation Evidence Record
| Field | Record |
|---|---|
| Question or decision | What is the evaluation being used to answer? |
| System identity | Model, scaffold, prompts or policies, tools, permissions, versions, date |
| Evaluation identity | Benchmark or task-set version, environment, grader, tests, parser |
| Target task population | What work is the result intended to describe? |
| Evaluation task frame | What tasks could enter the evaluation, and how were they selected? |
| Run policy | How are timeouts, crashes, retries, tool failures, interventions, and partial completion handled? |
| Observed evidence | Tasks, runs, artifacts, trajectories, outcomes, comparators |
| Scoring evidence | Does the scoring procedure recognize the intended outcome? |
| Generalization evidence | Task coverage, repetition, uncertainty, and sensitivity |
| Workflow evidence | What has been tested about performance under the intended operating conditions? |
| Decision criteria | Threshold, failure consequences, cost, alternatives, operational constraints |
| Supported conclusion | Strongest conclusion supported by the evidence above |
| Additional evidence needed | What study or validation would address the remaining uncertainty? |
The record keeps the evaluated system, evaluation conditions, available evidence, supported conclusion, and remaining uncertainty together without reducing them to one validity score.
References
- Kane MT. Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement. 2013;50(1):1–73.
- Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Medical Education. 2015;49(6):560–575.
- Zhu Y, Jin T, Pruksachatkun Y, et al. Establishing Best Practices for Building Rigorous Agentic Benchmarks. Advances in Neural Information Processing Systems. 2025.
- Yao S, Shinn N, Razavi P, Narasimhan K. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. International Conference on Learning Representations. 2025.
- Mizrahi M, Kaplan G, Malkin D, et al. State of What Art? A Call for Multi-Prompt LLM Evaluation. Transactions of the Association for Computational Linguistics. 2024;12:933–949.
- Rabanser S, Kapoor S, Kirgis P, et al. Towards a Science of AI Agent Reliability. Proceedings of the 43rd International Conference on Machine Learning. 2026.
- Raji ID, Denton E, Bender EM, Hanna A, Paullada A. AI and the Everything in the Whole Wide World Benchmark. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. 2021.
- D'Amour A, Heller K, Moldovan D, et al. Underspecification Presents Challenges for Credibility in Modern Machine Learning. Journal of Machine Learning Research. 2022;23(226):1–61.
- Kapoor S, Stroebl B, Siegel ZS, Nadgir N, Narayanan A. AI Agents That Matter. Transactions on Machine Learning Research. 2025.
- Shao J, Chen H, Zhang W, Pan M, Luo B. Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI. arXiv:2607.22368. 2026.