Biology offers abundant evidence that physical systems can learn, that is to say, physical systems can exhibit stable behaviour, conditioned on prior interactions with an external environment, in order to achieve a goal. We are entering an era in which learning machines can be engineered. In which case, what are the physical principles in play?
A learning machine can be instantiated in any physical system and not necessarily digital. Biological learning in brains is not based on algorithms running on digital computers, even if it can be simulated that way. What are the physical principles required for a machine to learn?
A learning machine, like any machine, is an open, dissipative physical system driven far from thermal equilibrium by access to a low entropy source of energy, for example, a battery. I will focus on simple classification in supervised learning. Here the objective is to learn a binary valued function, f(x), of the input data, x, by giving the machine a list of examples (x, f(x)) and adjusting the parameters of the machine through feedback so that the actual outputs are correct almost all the time. Error cannot be removed in a learning machine: it is an inherent feature of all learning. If you never make a mistake, then you never learn anything. if you only make mistakes , then you never learn anything either.
In a learning machine, reducing the error to zero in a finite machine would violate the laws of thermodynamics. The goal is to reduce the error probability, while making efficient use of the available thermodynamic resources.
A machine learning algorithm however is a mathematical procedure for approximating functions running (usually) on a conventional CMOS based von Neumann computer. There are very many machine learning algorithms and the discovery of new ones proceeds at an incredible pace. I want to contrast algorithms run on computers to actual machines that learn by thermodynamic constraints. In many ways this reduces to the question of who or what sets the goal? Who or what sets the error function? In a learning machine the goals are ultimately set by thermodynamics (in an evolutionary setting). In contrast, in ML algorithms, the algorithm designer sets the goal.
I am interested in quantum machines operating at very low temperature (they are cheaper to run), in which case the goal is to learn by exploiting quantum noise. How can quantum noise be harnessed for efficient learning? I will pursue this approach in future posts.
Fermionik 