Not “free as in speech” or “free as in beer”, nor “free energy” in the sense of perpetual motion machines, zero point energy or pills that turn your water into petroleum, but rather a particular mathematical object that pops up in variational Bayes inference and in wacky theories of cognition.
In variational Bayes
Free energy shows up in variational Bayes as the negative of the ELBO, the evidence lower bound, which AFAICT means that this term is defined by the Kullback-Leibler divergence. Presumably an analogous term would pop up in non KL approximation.
As a model for cognition
This term, with the same (?) definition appears to pop up in a “free energy principle” where it is instrumental as a unifying concept for learning systems such as brains.
Here is the most compact version I could find:
The free energy principle (FEP) claims that self-organization in biological agents is driven by variational free energy (FE) minimization in a generative probabilistic model of the agent’s environment.
The chief pusher of this wheelbarrow appears to be Karl Friston.
He starts his Nature Reviews Neuroscience piece with this statement of the principle:
The free-energy principle says that any self-organizing system that is at equilibrium with its environment must minimize its free energy.
Is that “must” in
- the sense of moral obligation, or is it
- a testable conservation law of some kind?
If the latter, self-organising in what sense? What type of equilibrium? For which definition of the free energy? What is our chief experimental evidence for this hypothesis?
I think it means that any right thinking brain, seeking to avoid the vice of slothful and decadent perception after the manner of foreigners and compulsive masturbators, would do well to seek to maximise its free energy before partaking of a stimulating and refreshing physical recreation such as a game of cricket.
What does this mean, precisely? There are dozens of Friston papers with minor variations on the theme and it is not clear where to start. I would recommend the clearer explanation in Millidge, Tschantz, and Buckley (2020) which summarises many of them and provides smoe extensions.