85% of companies burned by an AI mistake are racing to cut the humans who might catch the next one

Companies that suffered production failures from AI systems despite passing evaluations are paradoxically accelerating plans to remove human oversight from deployment decisions. New research from VB Pulse reveals this counterintuitive trend among enterprises already burned by evaluation failures.

The data shows 85% of companies that experienced AI mistakes in production are moving faster toward removing humans from the loop, not slower. This happens even as overall trust in automated evaluation metrics climbs. In July, 13% of 108 enterprises surveyed reported trusting automated evaluation systems, up from just 5% in June. The trust increase contradicts the logical expectation that failure would breed caution.

The phenomenon reveals a deeper problem in enterprise AI deployment. Companies developing evaluation frameworks believe their systems will work better next time, despite recent evidence to the contrary. Rather than adding guardrails after a failure, teams double down on automation. This creates a vicious cycle where confidence in metrics outpaces actual reliability.

The gap between evaluation success and production performance persists because enterprises often optimize evaluations for speed and cost rather than accuracy. A system can pass carefully constructed benchmarks while failing against real-world complexity. Companies learn this lesson painfully, then incorrectly conclude the solution is faster automation rather than better validation.

This trend matters because removing human review accelerates time to market but increases failure risk. Each removed decision point eliminates a chance to catch problems before customers encounter them. The cost of a production failure typically far exceeds the cost of human review time.

The data suggests enterprises view human oversight as a bottleneck to eliminate rather than a safety feature to preserve. As AI agents grow more autonomous and consequential, this approach becomes riskier. Companies racing to remove humans from deployment decisions after evaluation failures are essentially tripling down on the strategy that failed them initially. The research indicates enterprises still believe better automation is the answer to automation failures, not better human judgment.

CATEGORY