By 2017, the theory was settled. Neural networks worked. The Transformer architecture was elegant. Deep learning had proven itself on image recognition, game playing, and increasingly on language tasks. But there was a gap between "this works in a lab" and "this works on your laptop."

The gap was practical, unglamorous, and enormous. How do you gather enough text to train on? How do you store petabytes of data? How do you process it fast enough? How do you make the resulting models accessible to millions of people instead of just researchers with supercomputers?

The answers came from engineers solving boring problems at scale. Infrastructure builders, not theory innovators. People who made the math actually work in the real world.

The story starts with something most people have never heard of: Common Crawl. In 2007, Gil Elbaz and a small team started an ambitious project. They would crawl the entire public web — every month — and make the data freely available. Not summaries. Not indexes. The actual content of billions of web pages, archived and accessible to anyone.

Common Crawl became a public good. A nonprofit foundation maintaining the world's largest open dataset of web content. Almost every major language model trained since 2015 has used Common Crawl data. GPT-3, BERT, LLaMA, Claude — all built on a foundation of freely available web crawls that Common Crawl made possible.

But having the data wasn't enough. You needed to process it. Then Jeff Dean and Sanjay Ghemawat at Google published a paper on MapReduce — a framework for distributed computing that could process massive datasets by splitting work across thousands of machines. The open-source community built Hadoop based on Google's papers. Suddenly universities and small companies could process data at scales that previously required Google-sized infrastructure.

Still, you needed to know what to do with all that data and compute. The conventional wisdom in 2017 was that better algorithms were the path forward. Then OpenAI tried something different. They just made the models bigger.

GPT-1 in 2018 had 117 million parameters. GPT-2 in 2019 had 1.5 billion. GPT-3 in 2020 had 175 billion. Each version wasn't fundamentally different architecturally. Each was just bigger — more parameters, trained on more data, using more compute. And each version was dramatically better than the last.

This seemed almost offensive to some researchers. Where was the innovation? Where was the intelligence in just adding zeros? It felt like brute force, not elegance. Like trying to achieve flight by building a bigger catapult instead of understanding aerodynamics. But it worked.

In 2020, Jared Kaplan led a team at OpenAI that proved why it worked. They published "Scaling Laws for Neural Language Models," showing mathematically how model performance improved with scale. The relationship was predictable, smooth, and showed no signs of stopping. It turned AI development from research into something closer to engineering. You could plan. You could budget. You could predict returns on investment.

This predictability made AI investable. Venture capitalists and tech companies poured billions into compute and data. If bigger reliably meant better, then the strategy was obvious.

But none of this would have been possible without the hardware revolution happening in parallel. Graphics Processing Units — GPUs — had been designed to render video games. They were good at one specific thing: doing many simple calculations in parallel. Matrix multiplication, over and over, very fast. It turned out that training neural networks was also mostly matrix multiplication, over and over.

NVIDIA, led by Jensen Huang, recognized this before almost anyone else. By the time the scaling laws made big models viable, NVIDIA had the hardware ready to train them. Jensen Huang became one of the richest people in the world. The company that made graphics cards for gamers became the infrastructure backbone of the AI revolution.

Then the open-source community made the tools accessible. PyTorch, developed primarily at Meta, became the standard framework for building neural networks. Hugging Face built a platform for sharing models and datasets, making it trivial to use state-of-the-art models without training them from scratch.

None of these infrastructure builders became as famous as the researchers who published breakthrough papers. Jeff Dean is known in engineering circles but not in public. The Common Crawl team is essentially anonymous. But they made it real. They turned math into infrastructure.

The pattern was consistent: publish openly, build tools freely available, create infrastructure as public goods. Google published MapReduce rather than keeping it secret. Common Crawl made web data freely available. Meta open-sourced PyTorch. This wasn't pure altruism — companies benefited from widespread adoption. But it was also genuinely different from typical corporate behavior.

Training GPT-3 required an estimated 1,287 MWh of electricity — roughly what 120 US homes use in a year. The compute costs for training frontier models reached tens of millions of dollars per training run. Only a handful of companies could afford to play at that scale. Power consumption became a bottleneck. Microsoft signed deals to reopen Three Mile Island's nuclear reactor to power AI training.

The engineers had done their job brilliantly. They'd built infrastructure that could train models with hundreds of billions of parameters. But they'd also created an ecosystem where 80% of AI resources flowed toward making existing approaches bigger, faster, more efficient. Very little went toward different approaches.

The infrastructure builders had brought the world into our laptops. They'd made theory practical, made research accessible, made AI available. The scientists wrote the score. The engineers built the concert hall and filled every seat. Now we need composers willing to write different music.