Note: the source report is a republished blog post by James Shore on measuring AI's impact on delivery speed. It is an opinion and methodology piece, not firsthand reporting, and it contains no reported results from a specific company trial beyond an illustrative example.
James Shore, writing on his Successful Software blog and republished by O'Reilly Radar with his permission, argues that delivery speed is not the same as productivity. He defines software development productivity as value produced divided by money spent. Speed, in his view, is only one component of that measure, because a team that ships faster may still be building the wrong thing.
Shore says there is no objective measure of software development speed. Lines of code and pull requests are poor metrics. Features, bugs, and stories vary too much in size to count, and estimates are unreliable because teams are usually overly optimistic. He suggests comparing a team against itself instead.
He describes four problems with that approach. First, the estimate ratios of teams do not stay consistent, and in data from one real company they form an "S" curve. Second, people work differently, so comparing individuals to each other adds noise. Third, estimates shift depending on whether people expect to use AI. Fourth, process changes made alongside AI adoption, such as streamlined request workflows and