How the World Got Wired · A Naval Office in Arlington

The Quiet Decade

You should know about a particular group of these scientists, because they are the ones who eventually broke the winter.

In the late 1970s and early 1980s, at the University of California in San Diego, two researchers named David Rumelhart and James McClelland began organizing a small group that was thinking, again, about neural networks. They were not promising that machine intelligence was a year away. They were not announcing breakthroughs in the New York Times. They were working, in a careful, mathematical way, on the question of how to make multi-layer neural networks actually learn — the very thing Minsky and Papert had said couldn’t be done.

The group they organized was called the Parallel Distributed Processing group, or PDP for short. (You will, again, notice that the internet world’s love of acronyms is shared by the AI world.) Researchers from several universities came to San Diego for long visits to work with them. They wrote papers. They held workshops. They argued about how brains might work, and whether computers could be made to work the same way.

A young British scientist named Geoffrey Hinton was part of the group. So was a young man named Ronald Williams. Together with Rumelhart, in 1986, the three of them published a paper in the journal Nature with a quiet, technical title: Learning representations by back-propagating errors.

The paper described an algorithm — a step-by-step computational procedure — for training neural networks with many layers. The algorithm was called backpropagation, or backprop for short. The mathematics was not new — versions of it had been derived independently by other scientists, including an American researcher named Paul Werbos who had written about it in his PhD thesis in 1974. But Rumelhart, Hinton, and Williams put the idea together, demonstrated that it worked on actual networks, and presented it to the broader scientific community in a clear and memorable way.

That paper, in 1986, is one of the foundation stones of the artificial intelligence revolution that is happening right now, while you are reading this book. Almost every system you have heard of — the chatbots, the image generators, the systems that help drive cars, the language models that can write essays — uses, somewhere in its inner workings, a descendant of the algorithm Rumelhart, Hinton, and Williams described.

In the last lines of the paper, in tiny print, there is an acknowledgement section. It lists the people who funded the work.

One of them is the Office of Naval Research.

That gray building in Arlington, Virginia, that almost nobody outside the research community has heard of, paid for one of the most important papers of the late twentieth century. The Navy.

All 24 chapters of How the World Got Wired · Library · All courses