What use is utility?
If we must use the expected utility maximizer model for humans, what is the utility we should use?
2025-06-05 — 2026-04-09
Wherein the Implied Utility Functions of Animals Are Considered in Light of Machine-Learning Optimisation, and Value Learning Is Introduced as a Framework for Inferring Preferences From Behaviour.
Because machine learning models so often optimise a loss function, there is a degree to which we, as practitioners, internalise a world of agents with something like the fixed utility of classical rational actor theory. So we might start to think of evolved organisms like humans doing that that, having a utility function I mean. We might think this possibly to a greater extent than the metaphor deserves.
This notebook is is about how we can fabricate such utility functions if we absolutely must. If we need to construct such a pretence for animals, what does the implied utility function look like?
cf ecology of mind, what are human values, bitter lessons in ERM…
1 Value learning
One answer comes from reinforcement learning. See value learning.
2 Instead of utilities
Note that some ML algorithms don’t need utilities, but might subsist on intrinsic motivation.
3 Vector valued reward
See multi-objective optimization for a discussion of vector-valued reward and the perils of scalarization.
4 Incoming
Garrabrant’s Geometric Rationality, esp
Geometric rationality is my name for a cluster of techniques and behaviors that tend to maximize the geometric expectation (or equivalently maximize the expected logarithm) of natural features (of the world or the self). Geometric rationality techniques include Nash bargaining, Kelly betting, Thompson sampling, and (arguably) Bayesian updating. The posts in this sequence are relatively self contained, but draw on common themes.
esp