Amazon's director of AGI autonomy has identified reliability, not raw capability, as the core blocker preventing enterprises from deploying AI agents at scale. Bryan Silverthorn, who leads multimodal agent training at Amazon's AGI lab after the company acquired Adept AI, presented data at VB Transform 2026 showing that 85% of enterprises pilot AI agents but only 5% ship them to production. The gap reflects a fundamental misalignment between what the industry measures and what businesses actually need.
Silverthorn argues that reliability requires breaking down into four distinct dimensions: consistency, robustness, predictability, and a fourth dimension that underscores the problem better benchmarks alone cannot solve. Current industry metrics focus on capability improvements—raw performance on standardized tests. Enterprises care about something different. They need agents that behave the same way across different inputs, handle edge cases without catastrophic failures, and produce outcomes they can anticipate before deployment.
The distinction matters because capability and reliability move on separate curves. A model can outperform competitors on benchmarks while remaining unreliable in production. An agent that answers 98% of queries correctly still fails catastrophically in enterprise workflows where that 2% translates to lost revenue, regulatory exposure, or damaged customer trust.
This explains why the piloting-to-production ratio remains so low. Companies test agents, confirm they work in controlled settings, then discover they cannot predict failure modes in real-world conditions. A chatbot that hallucinates convincingly in a lab environment becomes a liability when answering customer support tickets.
Amazon's position inside this problem is instructive. As both a cloud provider and AI developer, the company sees enterprises struggling to move beyond pilots. Better benchmarks would not solve this. Enterprises need transparency into when and why agents fail, techniques to constrain behavior within known operational boundaries, and methods to gracefully degrade