What an eval is, and how to build one

How do you know a prompt or model change made your AI feature better?

Evals · about 5 minutes

An eval is a test for an AI feature: real inputs, a right answer for each, and a check that marks pass or fail. Build it from your own failures and run it on every change. Then you'll see whether a change helped.

Sketch: from a box of real inputs, Xiaohei holds an answer card up to a cut-out marked right answer; chutes drop cards into pass and fail