In 1906, a Russian mathematician named Andrey Markov was reading poetry. Not for pleasure, though he may have enjoyed it, but to settle an argument about randomness and independence. He chose Pushkin's "Eugene Onegin" and started counting — vowels and consonants, their patterns, their rhythms. What he discovered would eventually help computers talk.
Markov found something elegant: you could predict the next letter in a sequence by looking only at the current letter, ignoring everything that came before. In Russian text, certain letters naturally followed others. The past didn't matter — only the present state. He called these "chains," and they worked because language isn't random. It has patterns, rhythms, predictable flows.
But Markov died in 1922, long before anyone thought to apply his poetry analysis to machines. His chains remained a mathematical curiosity, elegant but seemingly impractical. The bridge between Markov's poetry and modern AI would be built by an American genius who loved puzzles.
Claude Shannon arrived at Bell Labs in 1948 with a question that seemed simple but wasn't: how much information is actually in a message? Not the meaning — the information itself. He published "A Mathematical Theory of Communication," and in it, he did something remarkable with language.
Shannon played prediction games. He would show people partial sentences and ask them to guess the next letter. "The q is almost certainly followed by..." Well, "u" of course. Very predictable, very low uncertainty. But "The cat sat on the..." could be followed by many words. More uncertainty, more information. Shannon invented a measure for this uncertainty and called it entropy. High entropy meant surprise. Low entropy meant predictability.
Then he calculated something extraordinary: English text has only about one to two bits of entropy per letter. Meaning that if you know the context, you can predict most of what comes next. Language, despite feeling rich and creative and infinite, is actually highly predictable. Not perfectly predictable — that would be boring — but statistically predictable.
Shannon proved mathematically what Markov had shown in Pushkin: you don't need to understand what language means to predict what comes next. You just need to count what actually happens.
These two insights — Markov's chains and Shannon's entropy — sat waiting in mathematics journals, elegant theories with no obvious application. Then Bell Labs tried to build a telephone system that could understand speech, and everything changed.
The first attempts were exactly what you'd expect from smart people: hire linguists, have them write rules about grammar and pronunciation, teach the computer how language works. Build understanding into the machine. Phoneticians carefully documented every sound. Grammarians mapped sentence structures. The systems were sophisticated, rule-based, intelligent in design.
They failed miserably.
Then Fred Jelinek arrived at IBM in 1972 with a different idea. Forget understanding. Forget rules. Just count what actually happens and use statistics to predict what comes next. Use Markov's chains — the probability of each word following the previous words. Use Shannon's entropy to measure whether your predictions match real language. Don't try to teach the computer what language means. Just show it what language does.
The linguists were appalled. This was crude, mechanical, unintelligent. It ignored everything they knew about how language actually works — syntax, semantics, deep structure, meaning. Jelinek's response became famous:
He wasn't being cruel. He was being empirical. The rule-based systems, for all their linguistic sophistication, couldn't handle the messy reality of human speech. The statistical systems, for all their apparent stupidity, could. They counted n-grams — sequences of words — and built models: what word tends to follow what other words? They didn't know why "neural network" was more likely than "neural elephant." They just counted that it was.
Jelinek assembled a team at IBM that would transform the field. Lalit Bahl solved a critical problem: what do you do when you encounter a word sequence you've never seen before? You can't just assign it zero probability — that breaks the math. Bahl developed smoothing techniques that remain fundamental today, ways to estimate probabilities for rare events without breaking the model.
Peter Brown applied these techniques to translation. Instead of teaching computers grammar rules for English and French, he fed them the Canadian parliament's bilingual transcripts — thousands of sentences in both languages. The computer learned probabilities: given this French sentence, what English words are likely? It was crude compared to human translation, but it worked better than rule-based systems.
Robert Mercer refined the language models themselves, making them more accurate and efficient. He would later take these statistical techniques to Renaissance Technologies and become a billionaire applying them to financial markets instead of words. But that came later. First came the revolution in speech recognition.
The paradigm shift was profound and unsettling. For centuries, humans had believed that understanding required intelligence, that you needed to know what words meant to use them correctly. Jelinek's team proved otherwise. You could build systems that appeared to understand language without understanding anything at all. You just needed enough data and Markov's math.
The Bell Labs and IBM researchers of the 1970s and 80s weren't just building speech recognition systems. They were proving a principle that would define the next fifty years of artificial intelligence: you don't need to understand meaning, you just need to count patterns. That principle, which seemed to insult human intelligence, turned out to be the path to artificial intelligence.
Shannon had given them the measuring stick — entropy showed whether predictions matched reality. Markov had given them the method — chains of probabilities, each state depending only on the previous states. The Bell Labs and IBM engineers gave them scale — millions of words counted, billions of n-grams tallied, statistical models that actually worked on real human speech.
None of them got rich from this work. Mercer's billions came from finance, not language. The others lived comfortable academic and industrial lives — respected, well-paid, not wealthy. They published their findings openly. They trained students. They built a community. The knowledge they created became public infrastructure.
And the insight they proved — that counting beats understanding, that statistics could substitute for semantics — became the foundation everything else was built on. When Google later built search, when researchers later built neural networks, when OpenAI later built GPT, they were all standing on the same principle: you don't need to understand language to predict it. You just need to count what actually happens.
Markov counting vowels in Pushkin. Shannon measuring entropy in English. Jelinek firing linguists. Bahl smoothing probabilities. Brown translating without grammar. This was the foundation. Not glamorous, not intuitive, not even particularly respectful of human intelligence. But it worked.
The scientists wrote the score. The engineers proved it could be played. And now everyone plays it, whether they know the composers' names or not.