How do you evaluate an LLM feature, and why don't ordinary unit tests work?

Ordinary unit tests assert exact outputs, but LLM output is non-deterministic and there are many acceptable phrasings for a correct answer, so assertEquals on a generated string is brittle and misleading.

Instead you build offline evals against a golden set -- curated inputs with known-good expected behaviors -- and score with criteria like exact match on extracted fields, semantic similarity, regex or schema checks for structure, or an LLM-as-judge that grades answers against a rubric. You run these as a suite when you change prompts or models so you can catch regressions, and you complement them with online evals in production (thumbs up/down, sampled human review) because real traffic differs from your golden set. The key insight interviewers want is that you treat quality as a measured, statistical property with a benchmark you can regress against, not a boolean the way traditional tests assume.