The internet's free lunch is ending. Training data that powered the first generation of large language models came cheap and abundant. Now AI companies face a data crisis as the digital commons grows polluted with machine-generated text, rights holders demand compensation, and genuinely novel content becomes scarce.
The problem is real. Search results increasingly surface AI-written garbage. Publishers and authors withdraw permission. Scraping agreements tighten. The low-cost, high-quality text that made training blazing fast and affordable is drying up. AI companies must now either pay for data or find new sources.
This explains the reported activity around old books. Legacy publishers hold vast archives of out-of-print works. These texts are old enough to avoid modern copyright disputes yet substantial enough to provide millions of training examples. Buying them outright sidesteps the ethics debate and the legal risk. It costs money. But the alternative costs more.
The data scarcity problem extends beyond text. Nvidia's new robotics simulator addresses a parallel bottleneck. Teaching robots requires motion data, visual examples, and consequences. Real-world collection is slow and expensive. Synthetic simulation fills the gap. Nvidia's system generates training scenarios using video and motion capture to teach robots behaviors in virtual space before deploying them physically. This approach bypasses data collection entirely by manufacturing it.
Both moves reveal the same constraint. Early AI scaled through abundance. Now scaling requires strategy. Companies either buy scarce assets at premium prices, generate synthetic alternatives, or build proprietary data moats. None are the free ride of 2022.
This shift shapes what comes next. Smaller labs lose the ability to train competing models on scrapped data. Larger companies with capital capture advantage. Data becomes infrastructure. The open-source movement that thrived on permissionless training slows. Licensing and synthetic generation become competitive advantages. The era of cheap AI innovation narrows.