Learning Is Not Memorisation: The Energy Cost of Remembering Everything

There is but one goal for a biological learning machine: survive long enough to reproduce. Learning changes the odds in your favour only if you don’t spend so much time and energy learning you die young. Ultimately, the goals of evolution and thermodynamics are aligned.

A simple learning task is classification. For example, suppose some red berries you encounter are packed with energy, but some kill you. To distinguish the two kinds of berries requires taking careful note of a lot more than simply the colour of the berry. Perhaps the good red berries have clustered skins (raspberries), but the ones with smooth skins will kill you. Perhaps some smooth-skinned red berries are OK but not if they are overripe. To find the safe red berries you need to take account of many more features.

You could simply not eat any red berries, and die of hunger in the winter. Another way to approach this problem; try to remember all the local factors that were present the last time you saw a member of your tribe eat a berry and die. The ‘memory first’ approach requires a lot of experience and a very good memory. There might be a lot to learn and a lot to recall and that may take so much time that … well, you are deselected. The objective in learning is to avoid memorisation. You don’t need to recall every detail in every situation in which you encountered a red berry. You only need to recall the patterns associated with good berries, and the patterns may be relatively easy to recall … if you can find them efficiently.

The kind of memorisation I am talking about is not how learning agents do memory. We tend to think our memories are like records in the old, analogue world, simply a photo or movie reel, that ‘we’ view in the Cartesian theatre. Our memories are not like that. If they were, it is doubtful we could function at all, like Funes the Memorious in the Borges story. The memories of a learning machine are reruns of learned predictive simulations, and every re-run is a new, different experience.

It takes a lot of resources to memorise every instance of a problem and a lot of time to search the data to know what to do in a novel encounter with a similar problem. In general the effort required, in both time and space, grows exponentially with every instance of the problem, even if the output is simply yes or no. For example, suppose you need to answer all possible 20 yes/no questions before you can decide if the outcome is good or bad. There are 2 raised to the power of 20 such inputs and each one could give two possible outcomes. That means to memorise this function you need to store

sets of yes/no values in order to have a complete look-up table. That is a very big number. Yet the key feature you need to learn might simply be that the good output occurs, almost every time, when all the twenty questions have an equal number of yes answers as no questions (even parity). You don’t need to memorise every instance of the problem, you only need to learn a particular feature of the problem that gives a low chance of making a mistake. And that may take a lot less effort to acquire … if you are a learning machine and are faced only with binary valued functions of many binary inputs.

The thermodynamics of this problem are important. If you simply memorise all instances of the problem you need to set aside locations in memory for 2²⁰ bits. Yet if you learn the key feature of the problem , say all you need to know is if the input has even parity , then you can store much less data.

If all inputs are equally, likely then this means the average entropy of what you need to store has decreased dramatically. In a physical machine (like you) entropy reduction like this requires that you pay a thermodynamics price by emitting enough heat into the environment to raise its entropy by as least as much as your learning machine has reduced your internal entropy. This is why all learning necessarily requires heat generation. This is simply a restatement of the Landauer erasure cost, if that means anything to you. Learning offers a huge evolutionary advantage.

Current AI (based on conventional silicon chips) learns well at a huge thermodynamic cost. Is it the minimum cost required? Absolutely not. If it were, no biological learning machine would ever have evolved on Earth. What is the minimum cost? AI engineers are trying hard to figure this out before they go broke paying the power bill. (That is what natural selection means in silicon valley. ) I think the answer is quantum.

Quantum — Noise and Error: Why Some Uncertainty Can’t Be Removed

Every physical experiment is subject to some uncertainty. Repeated measurements on physical systems prepared in the same way necessarily do not give identical outcomes. The results fluctuate.. This is noise. If the measurements are designed to verify that the preparations are identical, fluctuations will ensure that some give YES, but a few will give NO. These are errors. If the systems in questions are quantum systems, fluctuations in the measurement results is called quantum noise.

If all noise refers to fluctuations of measurement results why should we distinguish quantum noise from classical noise? The difference is subtle. In the classical case ( the physics of Newton, Maxwell, Boltzmann…), we assume that, with enough effort, we can make noise and error arbitrarily small, for example by lowering the temperature. In the quantum case this is impossible. The quantum world is a source of irreducible uncertainty. Fortunately quantum theory shows us how to manipulate the odds to our advantage. We have many more levers available than simply lowering the temperature. Quantum technology is the business of engineering those levers to control noise by directly intervening in the quantum world. The discovery of quantum mechanics is the discovery of new ways to intervene in the physical world.

Learning is impossible without noise and error. It should come as no surprise that quantum noise enables new kinds of learning machines. I will return to this in the next post.

Machine Learning or Learning Machine? The Difference Physics Makes

Biology offers abundant evidence that physical systems can learn, that is to say, physical systems can exhibit stable behaviour, conditioned on prior interactions with an external environment, in order to achieve a goal. We are entering an era in which learning machines can be engineered. In which case, what are the physical principles in play?

A learning machine can be instantiated in any physical system and not necessarily digital. Biological learning in brains is not based on algorithms running on digital computers, even if it can be simulated that way. What are the physical principles required for a machine to learn?

A learning machine, like any machine, is an open, dissipative physical system driven far from thermal equilibrium by access to a low entropy source of energy, for example, a battery. I will focus on simple classification in supervised learning. Here the objective is to learn a binary valued function, f(x), of the input data, x, by giving the machine a list of examples (x, f(x)) and adjusting the parameters of the machine through feedback so that the actual outputs are correct almost all the time. Error cannot be removed in a learning machine: it is an inherent feature of all learning. If you never make a mistake, then you never learn anything. if you only make mistakes , then you never learn anything either.

In a learning machine, reducing the error to zero in a finite machine would violate the laws of thermodynamics. The goal is to reduce the error probability, while making efficient use of the available thermodynamic resources.

A machine learning algorithm however is a mathematical procedure for approximating functions running (usually) on a conventional CMOS based von Neumann computer. There are very many machine learning algorithms and the discovery of new ones proceeds at an incredible pace. I want to contrast algorithms run on computers to actual machines that learn by thermodynamic constraints. In many ways this reduces to the question of who or what sets the goal? Who or what sets the error function? In a learning machine the goals are ultimately set by thermodynamics (in an evolutionary setting). In contrast, in ML algorithms, the algorithm designer sets the goal.

I am interested in quantum machines operating at very low temperature (they are cheaper to run), in which case the goal is to learn by exploiting quantum noise. How can quantum noise be harnessed for efficient learning? I will pursue this approach in future posts.