Karpathy sets out how the 1970s could have predicted language models
Andrej Karpathy, co-founder of OpenAI
Karpathy traced a chain of inferences, from digital knowledge work to scaling, that he said could have led an observer in the 1970s to anticipate something like modern language models.
In the Daily Brief of 5 November 2026
Andrej Karpathy argued on X that the arrival of large language models could have been reasoned out in advance. His chain runs from digital knowledge work to the internet, then to big data, then to statistical learning over neural networks, and finally to scaling.
An observer following those steps in the 1970s, he said, would have expected something like today’s language models rather than hand-programmed robotics. Researchers have shared the thread widely as a historical framing.