Evaluations
Session
Generative AI
Decision-Driven Evaluation for Generative AI
Monday Jun 1 / 03:40PM EDT
Evaluation should not just produce metrics, which may turn out to be “metrics of convenience”, but should directly inform product and engineering decisions.
Terran Melconian
Principal Applied Scientist - AI @Zillow, ML and Gen AI, Former Data Science Teacher & Engineering Director
Session
Evaluations
Building Reusable Evaluation Frameworks for Agentic AI Products
Tuesday Jun 2 / 11:30AM EDT
This talk covers methods of evaluating AI Agents, with an example of how we built evaluation frameworks for a user-facing AI Agent system that has been in production for almost two years.
Susan Chang
Principal Data Scientist @Elastic