Follow AES events Research & Commons
Evaluation Science Briefs
Concise, source-linked articles on how to design, interpret, and use evidence from AI agent evaluations.Explore AES research and resources →Featured / Start here
How Close Is an Agent Evaluation to the Real Workflow?
When does benchmark performance support a conclusion about the workflow where an agent will actually be used?
Read brief- 01
What Does an Agent Evaluation Actually Support?
Observed runs, evaluation results, target-setting performance, and decisions require different evidence.
- 02
How Do You Know Your Agent Grader Works?
Tests, rubrics, parsers, and LLM judges need validation before their scores can support comparisons between agents.
- 03
Is This Agent Result Reproducible?
Repetition under one configuration cannot show whether a result survives changes in tasks, prompts, tools, environments, implementations, or time.
- 04
Is This Agent Good Enough for the Decision?
A score supports a decision only when uncertainty, thresholds, failure costs, alternatives, and the structure of errors are explicit.
- 05
What Should an Agent Evaluation Report So Someone Else Can Use It?
A reusable evaluation report identifies the system, task frame, run policy, grader, uncertainty, target claim, decision context, and limits of the evidence.