Agenda discussion
A discussion of open questions and priorities that could shape future evaluation research.
Summary available after the symposiumFall 2026 · New York City
Agent Evaluation Science
Symposium
A one-day symposium on the science and practice of evaluating AI agents.
- Date
- Friday, November 20, 2026
- Location
- Downtown Manhattan · New York City
Registration is open to the public; acceptance prioritizes presenting authors.
Explore the symposiumScientific evaluation◆Construct validity◆Measurement reliability◆Benchmark integrity◆Agent safety◆Grader design◆Failure analysis◆Evaluation infrastructure◆Real-world evidence◆Reproducible systems◆
Symposium program
Bringing evaluation questions into one room
Agent evaluation is developing across research, industry, and different application domains, often around separate methods and problems. The symposium brings these perspectives together to compare evidence, surface disagreements, and identify priorities for future evaluation research.
Evaluation science agenda · Program format
0104
Talks & fireside
Invited researchers and practitioners will share current work, open problems, and perspectives on where agent evaluation is heading.
View speakers →02Contributed research
Selected papers, extended abstracts, and posters presenting ongoing and published work in agent evaluation.
View the call →03Evaluation in practice
Cases and lessons from evaluating agents in real-world settings and workflows.
View program →Call for presentations
Submit work on agent evaluation
Submit evaluation studies, audits, negative results, failure analyses, tools, datasets, production lessons, and real-world evaluation cases. Ongoing and previously published work can also be presented through the extended abstract track.
- Evaluation studies
- Audits and replications
- Negative results
- Failure analyses
- Tools and datasets
- Production lessons
- Real-world evaluation cases
Industry and real-world submissions are assessed on documented evidence, production lessons, and findings relevant beyond a single system or setting. A new benchmark or algorithm is not required.
Submissions open
August 252026Abstract registration deadline
October 202026Full submission deadline
October 252026Notification of acceptance
November 52026Symposium
November 202026Affiliations represented
Speaker and committee affiliations
- Agent Evaluation Science
- Harvard University
- Massachusetts General Hospital
- BRIDGE GenAI Lab
- Princeton University
- Cornell University · Cornell Tech
- The University of Texas at Austin
- University of Technology Sydney
- Australian Artificial Intelligence Institute
- University of Alberta
- Amii
- Meta Superintelligence Labs
- Sony AI
- Raycaster
- Google DeepMind
- Massachusetts Institute of Technology
- MIT Media Lab
- University of Massachusetts Amherst
- Max Planck Institute for Intelligent Systems
- Columbia University
- DAPLab
- New York University
- Cohere Labs
- J.P. Morgan
- IBM Research
- Microsoft Research
- University of Illinois Urbana-Champaign
- Dartmouth College
- Vector Institute
Affiliations describe speakers’ and committee members’ institutional or organizational connections. They do not indicate sponsorship.
Agent Evaluation Science Fall 2026Friday, November 20 · Downtown Manhattan, New York City
Meet the committee
