A startup called AI IQ has launched a website that assigns estimated intelligence quotients to over 50 frontier language models and plots them on a standard bell curve, mimicking the human IQ testing framework. The interactive visualizations at aiiq.org have gone viral on social media in recent days.
Enterprise technologists have praised the charts for making an opaque and complex AI market more understandable at a glance. The visual comparisons offer a quick way to rank competing models like OpenAI's GPT-4, Anthropic's Claude, and Google's Gemini against each other.
However, researchers and AI commentators have pushed back hard against the entire premise. The criticism centers on a fundamental problem: IQ tests measure specific cognitive abilities in humans within a defined population. Applying that framework to AI models fundamentally misrepresents what these systems do. Language models lack the general intelligence that human IQ tests attempt to capture. They excel at pattern matching and statistical prediction on text, not at reasoning, common sense, or embodied understanding.
The bell curve visualization itself creates false equivalences. It suggests a linear spectrum of intelligence when AI capabilities are actually multidimensional and task-specific. One model might excel at coding while another performs better on reasoning tasks. A single IQ-style number erases these crucial differences.
There's also the problem of test selection bias. Whatever benchmarks AI IQ uses to calculate scores will inevitably favor certain model architectures or training approaches over others, making the rankings less objective than they appear.
The tension here reflects a broader challenge in AI evaluation. The industry desperately needs standardized ways to compare models, but IQ-style scoring risks oversimplifying and misleading users. Enterprise buyers want clarity, but clarity built on a flawed foundation can lead to worse decisions than transparency about genuine complexity.
AI IQ's charts are intuitive and shareable.