Coding agents are being sold on near-perfect scores, and the Co-Design Bench results should end that comfort. According to the report, GPT-6 Astra, the best of ten models tested, meets about 70 percent of a task's requirements on average, yet is fully correct in only 26.4 percent of tasks. Anyone who has handed a half-finished feature back to a developer knows what that gap means in practice: the agent does most of the work and someone still has to finish it by hand.
The authors have a fair case against the existing tests. The report says that when models can run, check, and fix their own code, they reach scores above 99 percent on older benchmarks, leaving little room to tell models apart. Building the new test from real user change requests, including the ones where users had to rephrase or roll back, is the right instinct. A benchmark that reflects where agents actually struggle is more useful than one that rewards tidy, self-contained problems.
The grading design also deserves credit. Points go only to features that work in a browser, and a second AI judge re-checks every positive verdict for mock-ups or hardcoded results. Strict checks like these are what separate a demonstration from a deliverable.
The most useful finding is the one about the environment. The report notes that Qwen3.8-27B scored either 75 or 56 percent on larger overhauls depending on the harness, and that which model came out ahead changed with the setup. A leaderboard that names only the model is therefore incomplete. Companies choosing a coding agent should test the model and its working environment together, because the pairing is what they will actually deploy.
The limits are real, and buyers should keep them in view. The test uses one fixed request per task and leaves out the follow-up corrections that define real sessions, and the AI judges can err. Still, the direction is clear: read benchmark scores as a starting point, and budget for human cleanup on most tasks until the numbers say otherwise.