2024 physics Nobel for work on artificial neural networks | Explained Premium

Hopfield and Hinton's groundbreaking work in artificial neural networks paved the way for modern AI applications like ChatGPT.

The story so far: On October 8, John Hopfield and Geoffrey Hinton won the 2024 Nobel Prize for physics “for foundational discoveries and inventions that enable machine learning with artificial neural networks”. Their work lies at the roots of a large tree of work, the newest branches of which we see today as artificially intelligent (AI) apps like ChatGPT.

An accessible AI today is likely to be an implementation of an artificial neural network (ANN) — a collection of nodes designed to operate like networks of neurons in animal brains. Each node is a site where some input data is processed according to fixed rules to produce an output. A connection between nodes allows them to transfer input and output signals to each other. Stacking multiple layers of nodes, with each layer performing a specific task with great attention to detail, creates a machine capable of deep learning.

The popular imagination of AI today is in terms of computing: AI represents what computers like those in smartphones can do today that they weren’t able to yesterday. These abilities are also beginning to surpass what humans are capable of. So it is a pleasant irony that the foundations of contemporary AI, for which Hopfield and Hinton received this year’s physics Nobel Prize, are in machines that started off doing things humans were better at — pattern recognition — and based on ideas in statistical physics, neurobiology, and cognitive psychology.

In 1949, Canadian psychologist Donald Hebb introduced a neuropsychological theory of learning to explain the ability of connections between neurons to strengthen or weaken. Hebb posited that a connection, or synapse, between two neurons becomes more efficient if the neurons constantly talk to each other. In 1983, Hopfield developed an ANN whose nodes used Hebb’s postulate to learn by association. For example, if a node is exposed to many texts, one set in English and the other its Tamil translation, it could use Hebbian learning to conclude “hand” and “kai” are synonymous because they appear together most often.

Another distinguishing feature of a Hopfield network is information storage. When the network is ‘taught’ an image, it stores the visual in a ‘low-energy state’ created by adjusting the strengths of the nodes’ connections. When the network encounters a noisy version of the image, it produces the denoised version by progressively moving it to the same low-energy state. The use of ‘energy’ here is an echo of the fact that the Hopfield network is similar in form and function to models researchers have used to understand materials called spin glasses. A low-energy state of a Hopfield network — which corresponds to its output — could map to the low-energy state of a spin glass modelled by the same rules.

Hopfield’s mapping was a considerable feat because it allowed researchers to translate ideas from statistical physics, neuropsychology, and biology to a form of cognition.

Hinton’s share of the Nobel Prize is due to his hand in developing the first deep-learning machines. But as with Hopfield standing on Hebb’s shoulders, Hinton stood on those of Ludwig Boltzmann, the Austrian physicist who developed statistical mechanics. In 1872, Boltzmann published an equation to predict, say, the possible behaviours of a tub of fluid with one end hotter than the other. Whereas the first guess of a simple logic would be that all the possible states this system can take would be equally probable, Boltzmann’s equation predicts that some states are more probable than others because the system’s energy prefers them.