Expedia's chief AI officer Xavi Amatriain argues that evaluations have replaced traditional product requirement documents in AI development. Speaking at VB Transform 2026, Amatriain stated that teams now encode product specifications directly into evals, which test system behavior across security, functionality, and edge cases.
The shift reflects how AI-generated code development works in practice. Rather than writing detailed PRDs upfront, engineers design comprehensive evaluation frameworks first. These evals include red teaming tests, security checks, and functional requirements. Developers then build code to pass these tests, flipping the traditional waterfall process on its head.
Amatriain emphasized that AI-assisted coding amplifies this trend. When systems like Claude or GPT-4 generate code, the quality depends entirely on what you ask them to do. Clear specifications in eval form guide code generation far more effectively than narrative documents. The engineer's work shifts from implementation to defining what success looks like.
This approach carries real advantages. Security requirements embed into product design before code exists. Teams catch edge cases and failure modes earlier. Evals become living documentation that stays current with code, unlike static PRDs that drift.
The approach also exposes risks. Poorly designed evals miss important behaviors. Over-reliance on automation skips human judgment about user needs. Teams must think harder about what to measure, not less.
Amatriain's background at Google, where he led AI infrastructure and ranking systems, shapes this perspective. Large-scale AI work at Google already operates this way. As AI coding tools mature, this pattern moves into mainstream development.
The claim that "all your thinking goes into evals" captures something real about how AI development works. The bottleneck shifts from coding to specification. Teams that excel at writing rigorous, comprehensive evaluations will ship better products faster. Those that treat evals as afterth