code red
Speaker

Terran Melconian

Principal Applied Scientist - AI @Zillow

Terran Melconian is a Principal Applied Scientist at Zillow, where they lead work on evaluation and quality for generative and agentic AI systems, as well as classical machine learning models. Their focus is building decision-driven evaluation frameworks that connect model behavior to real product outcomes and long-term user impact. Previously, they have held senior technical and leadership roles at Google, TripAdvisor, Meta, and early-stage startups, where they built data science and search teams from zero to production, owned high-availability distributed systems end-to-end, and drove measurable business impact through experimentation and optimization. They hold S.M. and S.B. degrees in Aeronautics and Astronautics from MIT, where their research focused on large-scale simulation of complex systems, and this continues to influence their work on evaluating AI systems operating under uncertainty.

Session

Decision-Driven Evaluation for Generative AI

This talk starts with an example of an evaluation suite which looked good: it was comprehensive and detailed, but we discovered that it did not provide the clarity we needed to make a launch decision.

Read more

Date

Monday Jun 1 / 03:40PM EDT ( 50 minutes )

Location

Metcalf Hall Small

Share