Optionality as an end in itself
On optimizing for not optimizing
2026-04-12 — 2026-07-21
Wherein the Moral Status of Preserving Future Possibility-Space Is Examined Through Three Formal Frameworks, Including Empowerment Theory, Ergodicity Economics, and Quality-Diversity Algorithms.
An intuition I took from Indy Johar’s recent Long Now essay on civilisational optionality1 is something like the following: what we ought to be optimizing — at least at civilisational scale — is not any particular state of affairs, but the space of states of affairs we could still reach in the future. Not expected utility from the likely states, but the ‘raw volume’ of futures still on the table at the time we next decide.
Is that a coherent moral aim? What kind of object is it? How does it relate to the diversity-as-end intuition, the empowerment formalism, the intrinsic motivation literature, or the asymptotic leviathan civilisational story? I don’t know!2
My research agent surfaced three practical formalisations, and a couple more intuitionistic variants. Let us unpack ’em.
1 Formal versions
By “formal” here I mean “we can compute this in principle, and someone has for at least one case”.
1.1 (Informational) Empowerment
(Informational) Empowerment (A. S. Klyubin, Polani, and Nehaniv 2005) is clean and simple-ish, under certain assumptions, e.g. in a stationary world, known up to aleatoric dynamics. The target is the channel capacity between an agent’s actions and its future sensory states, \(\mathfrak{E}(s) = \max_{p(a)} I(A;\, S')\). An empowerment-maximizing agent favours states from which many futures are reachable, and avoids states from which few are. That is a drive to keep options open, where options = entropy.
Caveat: Empowerment is defined over a fixed action space, a fixed state space, and a fixed (or at least learnable) transition kernel. In an open or non-stationary world — where the action space itself can grow, new states emerge, or the dynamics drift — the channel capacity \(I(A; S')\) ceases to be a well-posed quantity. Calculating it for any non-trivial horizon is rather punishing even in the closed, stationary case; in the open world, I don’t know what it would even mean.
1.1.1 Causal entropic forces
Wissner-Gross and Freer (2013) is an alternative but similar version that estimates a somewhat different objective (maximise entropy of my future path entropy conditional upon a single next action and thereafter following the dynamics), but it is in the same spirit: a drive to keep options open.
- A Grand Unified Theory of Everything
- dyth/causal-entropic-forces: Python reimplementation of experiment 1 from the paper.
1.2 Ergodicity economics
Ole Peters (2019) arrives at a similar place from a different direction. A system is ergodic if averaging an observable across many parallel trajectories at a single moment gives the same answer as averaging it along one trajectory over time — i.e., “many people now” and “one person across many years” agree.
Many systems we care about are not ergodic. Peters’ canonical counter-example is a coin flip bet: each round, my wealth \(W\) is multiplied by \(1.5\) if the coin lands heads and by \(0.6\) if it lands tails. The coin is fair; whether the gamble is fair depends on which average I care about.3
The ensemble average surveys many of us playing in parallel, at a fixed round \(n\). The mean multiplier per round is the arithmetic average of the two outcomes, so
\[\mathbb{E}[W_n] \;=\; W_0 \left(\frac{1.5 + 0.6}{2}\right)^{\!n} \;=\; W_0 \times 1.05^n .\]
On this average the gamble is better than fair: a 5% edge per round, compounding! That sounds like a free lunch.
The time average follows one of us (e.g. me) playing many rounds in sequence. The law of large numbers deals me heads on about half the rounds and tails on the other half, so
\[W_n \;\approx\; W_0 \,(1.5 \times 0.6)^{n/2} \;=\; W_0 \times 0.9^{n/2} \;\approx\; W_0 \times 0.95^n .\]
On this average the same gamble is worse than fair: my wealth decays by about 5% per round until I have nothing.
Both calculations are correct; they disagree because repeated multiplication can be understood in terms of the geometric mean of the multipliers, \(\sqrt{1.5 \times 0.6} \approx 0.95\), which is less than the arithmetic mean \(1.05\). That disagreement is all that non-ergodicity means here. After 100 rounds the ensemble mean is \(1.05^{100}\,W_0 \approx 130\,W_0\), while the median player holds \(0.9^{50}\,W_0\), about half a percent of their starting stake. The mean is propped up by an exponentially rare minority of heads-heavy histories. This is not reassuring to me as a player since I get to live only one history, and that is very unlikely to be the lucky one.
cf Garrabrant’s Geometric Rationality.
Peters argues that under non-ergodicity, expected utility optimizes the wrong average. In our coin flip with \(\mathbb{E}[\log r] < 0\) almost every trajectory decays exponentially, so the time average of wealth is \(0\), and in a gamble with \(\mathbb{E}[\log r] > 0\) it would be \(\infty\) instead. This is degenerate either way — useless for ranking one gamble against another. The observable with a finite, informative limit is the growth rate, \(\tfrac{1}{n}\log(W_n/W_0) \to \mathbb{E}[\log r] \approx \log 0.95\) per round. Almost every trajectory shares this growth rate. The growth rate, unlike wealth, is ergodic: its time average agrees with its expectation, so its expectation describes what happens to a single player, which is the property that prompted us to take expectations in the first place. Ergodicity economics says to optimize that number: expected wealth endorses the coin flip above, time-average growth refuses it. Under multiplicative dynamics the growth rate is an expected log, so this happens to coincide with log utility. Neatly, the log emerges naturally from the dynamics, rather than being given. Contrast an additive variant of the gamble, with stakes that do not scale with our wealth: heads wins us fifty dollars, tails loses us forty, whatever our balance. The gain per round is then the ergodic observable — \((W_n - W_0)/n \to \tfrac12(50) - \tfrac12(40) = 5\) dollars along almost every (solvent) trajectory, which is also its expectation — so ranking gambles by time-average growth is the same as ranking them by expected wealth. Maximizing time-average growth over how much we stake is the Kelly criterion, which disfavours any action that risks ejecting the agent from the support of viable trajectories. Optionality-flavoured behaviour falls out of this without anyone having to add “preserve options” as a separate goal.
::: ## Spicy take
Man, this ergodicity economics thing is not wrong per se but feels like an excessively galaxy-brained way to re-derive the known fact that linear utility in any single good is silly. Suppose I want to “have lots of money”— does this mean that I want to maximize my expected wealth, my expected log-wealth or my median wealth, or minimise the chance that my wealth goes to zero, or to really juice the 0.2-quantile of \(5W^2+\sqrt[3]{W}\), or whatever other statistic of the distribution? This depends. In general my utility is probably monotonic in money, but it would be surprising if it were linear in the good and indifferent to the tails. My “true” utility from money is not naturally well characterised by some dumb default option like \(U(W) = W\) or \(U(W) = \log W\), it is some idiosyncratic function of the distribution of wealth, maybe even wealth over time. Assuming a linear dollars-to-utility conversion is something they teach you not to do in microeconomics; it’s a degenerate and unrealistic model producing implausible behaviour for the meat sack humans. Anyway, if we care about any interesting quantile of the wealth distribution, we also learn not to take stupid bets that go to zero, without needing to invoke any ergodicity argument. Don’t get me wrong, I think ergodic arguments are important in various ways, but I am not convinced they add substantially to an already enormous pile of reasons that humans shouldn’t take insane bets.
Am I missing something? :::
1.3 Quality-diversity algorithms
In a textbook evolutionary algorithm, under a fixed goal, the algorithm typically converges to some optimum: after enough generations, every member of the population is a mildly mutated variant of the reigning champion. Evolutionary algorithms are still optimization, after all, and diversity is the first casualty of optimization. The field maintains a sub-literature of patches — niche-construction, fitness sharing, novelty bonuses — to slow the collapse.
Quality-diversity algorithms promote diversity using an explicit diversity goal. Rather than returning the single best solution, they return an archive, partitioning the space of behaviours into niches and keeping the best solution found in each. MAP-Elites (Cully et al. 2015) does that, imposing a grid of pseudo-niches on behaviour descriptors; novelty search (Lehman and Stanley 2011) drops the quality term altogether and rewards only behaviours the archive has not seen before. This is nominally inspired by biology: diversity survives natural selection, when it does, because there is no single prize to converge on — a landscape of distinct niches is a landscape of distinct local optima, and the best beetle and the best whale are not competing in the same slot. MAP-Elites hard-codes that, with grid cells.
There is an overt informal motivation in terms of option value. In Cully et al. (2015), a hexapod that breaks a leg need not relearn locomotion from scratch; it can consult its pre-computed archive of qualitatively different gaits until it finds one that still works with 5 legs. The archive is insurance against an environment shift that the training objective never anticipated. Formally, this cashes out as a static diversity measure over currently-realized behaviours, evaluated at a snapshot.
Strictly this is diversity-as-end; there is no optionality, since the algorithm is not forward-looking. As such it kinda sneaks in to this taxonomy, as the only formalism of the three whose maximand we can compute directly because it is not actually about future options, but I’d already written the previous paragraphs when I finally worked that out.
1.4 What did that get us?
Hmm, did we sketch out a conceptual space there? I feel like we just sampled some cool ideas and got nowhere.
- Empowerment maximizes mutual information from actions to futures, at least in the agent’s own model of the world, which it somehow knows.
- Ergodicity economics minimizes the probability of leaving the support of viable trajectories, by re-averaging a single time series the “right” way.
- Quality-diversity maximizes entropy over an archive of currently-realized behaviours, treating diversity as a hedge against an unknown future fitness function.
All three put an entropy-like quantity in the place where standard utility would have put a single-target loss. They differ on what the entropy is over — futures, trajectories, or current behaviours.
I’m skeptical we can even “solve for optionality”, at least in any meaningful sense, in a world where the action space itself can grow, new states emerge, and the dynamics drift.
The problem of calculating optionality is not just difficult in an open world; it seems to be ill-defined, or if well-defined then intractable. So any claim that we should be optimizing for optionality will naïvely cash out as a claim that we should be optimizing for some proxy for optionality, which is just another objective, innit? Is anything especially good about such proxies compared to the default?
Hell, isn’t the desire for wealth already a proxy for optionality, in that it is a hedge against (some large class of) unknown futures?
2 Intuitive versions
God help you if you want to compute these bad boys.
2.1 Moral uncertainty
If we do not know what is good, we should not lock in any one answer. This is one motivation for Bostrom’s long reflection (Bostrom 2014), and behind milder claims that we should not race to build single-objective superintelligences before we have finished arguing about what their objective should be (and maybe, how to regularize it?). There is a formal moral-uncertainty framework (MacAskill, Bykvist, and Ord 2020) for how to act when uncertain across moral theories; explicit optionality is one possible response to that uncertainty rather than the only one. Optionality here is the meta-property that increases our chances that if we ever figure out what the real objective is, we are still in a position to act on it.
This one is very popular amongst those already committed to the potential to bring about superintelligences which might need to act using a specific utility that we have not yet figured out.
2.2 Antifragility
Taleb’s antifragility (Taleb 2013) argument is that some systems gain from disorder, where exposure to small shocks improves long-run resilience. Scott Alexander wrote about diversity, libertarianism, and corporate censorship along these lines. Antifragility is not quite the same as optionality — antifragile systems benefit from volatility, whereas option-preserving systems merely refuse to foreclose — but they share an aversion to lock-in and a fondness for redundancy. Also Taleb can spin a yarn and coin a sticky metaphor, so this one is here to stay, even if it’s hard to pin down anything helpfully formal.
2.3 Diversity
Plain old vanilla diversity is closely adjacent but not identical. Diversity-as-end is about the configuration space of possible humans, possible cultures, possible intelligences now. Optionality is about the configuration space of possible futures from now. Diversity is a static observable; optionality is a forward operator on diversity — how much of it will still be available in \(T\) steps?
These could come apart in cases like a homogeneous society that nonetheless preserves the option to diversify, or a maximally diverse society that has, through some narrowing of common infrastructure or language, foreclosed the ability to recombine. I’m skeptical that the former is realizable (Maybe Tokugawa Japan under the sakoku policy?); the latter is roughly what some critiques of platform monocultures claim is happening to us.
3 Why might optionality be a moral good?
A few candidate arguments, in increasing order of how much weight they bear:
- Instrumental. Optionality is a low-regret proxy for whatever the good turns out to be. If we cannot identify the target, preserving the ability to aim is the next best thing. Optionality lets us defer the hard moral philosophy to some poor future bastard.
- Aggregative. There are many possible goods, they are not commensurable, and we cannot pick one. The union of futures realizing different goods is therefore better than any single future, in something like the Dixit-Pindyck sense of option value (Dixit and Pindyck 1994), applied to ethics rather than capital budgeting. This crops up in environmental economics of natural resources: the preservation value of an ecosystem includes the option value of being able to use it later under future preferences and information we do not yet have (Weisbrod 1964; Krutilla 1967; Arrow and Fisher 1974).
- Constitutive. The open-endedness of possibility is itself the thing we value. A frozen optimum, even an optimal one, is dead in some intuitively important sense. Is not being open-ended somehow a “good”? Seems important.
4 Compared with utilitarianism
Let us suppose I have committed to optionality as a moral good. How does this distinguish my position from utilitarianism? Both are consequentialist ethics — actions are evaluated by their downstream effects on the world. However, we disagree about which downstream effects matter. Deontological and virtue-ethics objections to consequentialism apply equally to both ofc.
Thoughts:
First, as noted above, optionality can be made to look like a particular utility function, or a regularizer on a utility function. Define \(U(s) = \log |\text{reachable futures from } s|\), or some entropy-like proxy thereof, and a maximiser of \(U\) is a maximiser of optionality. So we could call this just a special utilitarianism, with a specific if weird utility function. The empowerment literature does flag this — empowerment is sometimes called a pseudo-utility, with some nice properties wrt generalizing to changing or under-specified objectives.
So, it is utility with a structural commitment to how it aggregates over futures: diminishing returns for piling up additional futures and an increasing penalty for foreclosing any as we run low on futures. That seems … fine? Consider some alternative aggregations: maximin (no improvement elsewhere compensates any worsening of the worst case), Kelly criterion (maximise the expected log of wealth), and expected utility (maximise the expected value of wealth). Kelly/log implies smooth substitution everywhere except at the ruin boundary, at which point it gets arbitrarily aversive (we try really hard not to die). These objectives are all on the spectrum of generalized means: linear utilitarianism aggregates futures by the arithmetic mean (\(p=1\), perfect substitutability), maximin is the \(p\to-\infty\) limit (none), and log/Kelly/optionality utilities sit at the geometric mean (\(p\to 0\)), implying partial substitutability, hardening to refusal at the boundary.
Second: aggregation across agents. Utilitarianism’s (default) aggregation between agents is linear: \(U_{\text{total}} = \sum_i u_i\). The agents are commensurable; my utility trades for yours one-for-one. Information-theoretic optionality does not aggregate that way, and two toy channels show why not. Synergy: suppose the future is the XOR of two agents’ actions, \(S' = A_1 \oplus A_2\). Alone, each of us is powerless — whatever I do, the future remains a fair coin — but together we determine it completely. Redundancy: suppose agent 2 merely echoes agent 1. Then each of us looks fully empowered on our own, yet jointly we control no more than either did alone. So in general \(I(A_1, \ldots, A_n;\, S') \neq \sum_i I(A_i;\, S')\), and which side is bigger is a fact about the world, not a moral choice. Worse, the summands on the right are not even well-defined: my empowerment \(I(A_i;\, S')\) depends on what everyone else is doing, because the other agents are part of my environment — in the XOR world I am powerless against a random partner and fully empowered against a predictable one. There is no canonical “we” to aggregate over without first fixing a convention for everyone’s behaviour; the parts do not exist prior to the whole. We can force linear aggregation by fiat — fix such a convention, assign each agent a scalar empowerment under it, and add them up — but at that point we are doing utilitarianism with a particular intermediate quantity, not optionality at the collective level. Whether the natural sub- or super-additivity of joint information is closer to the moral truth than utilitarianism’s linearity is not at all clear to me. Has someone written about this? Surely someone has written about this.
Partially: partial information decomposition (Williams and Beer 2010) is the formalism for splitting joint information into synergistic and redundant shares, and coupled empowerment maximization (Guckelsberger, Salge, and Colton 2016) deploys multi-agent empowerment to steer companion NPCs in games, though neither of these is exactly concerned with ethics. The general problem of attributing a non-additive joint quantity linearly to its contributors is the Shapley value (Shapley 1953); treating collective empowerment as a cooperative game in that sense looks like a paper someone should have written, but I have not found it.
Anyway, as such optionality evades some of utilitarianism’s classical headaches. It does not produce a Parfit-style repugnant conclusion: it has no built-in preference for vast populations of barely-distinct futures over small populations of richly-distinct ones. It does not endorse extinction by negative-utilitarian logic, because extinction has measure zero in reachable-futures space.
It also might dodge the standard form of Pascal’s mugging — the longtermist worry that tiny probabilities of astronomically large future utilities should dominate present moral reasoning. Pascal’s mugging gets its leverage from the multiplicative structure of expected utility (\(EV = P \times U\), with \(U\) unbounded above). Optionality measures are typically bounded — channel capacity by \(\log |S’|\), archive entropy by archive size — and the ergodicity-economics move explicitly rejects ensemble-average reasoning that lets tail outcomes dominate. Tiny-probability astronomical futures simply do not get the same purchase on the maximand. Tarsney (2025) makes a parallel move within utilitarianism itself, capping expected-value reasoning under sufficiently high background uncertainty; the structural worry is similar but the bound is imposed as a side constraint rather than built into the framework.
5 Optionality catastrophes
Empowerment, at least in some form, is one of the convergent instrumental goals that the AI-safety literature worries about — Omohundro’s basic AI drives (Omohundro 2008), Turner et al. on optimal policies seeking power (Turner et al. 2021). An agent that preserves its own future optionality can, by the same arithmetic, be foreclosing ours. “Keep options open” is symmetric across agents only when there is no resource competition; there is, so it isn’t. The information-theoretic version of “live and let live” can still end up looking like imperialism if we give it enough time and compute. And even if optionalities are not linearly summable, we can still argue about the weighting.
Also, optionality metrics on their own only oppose dystopia-bound trajectories if dystopia in fact has low reachable-future-volume. That is plausibly true for some dystopias and less obviously true for others — a stable totalitarian regime with lots of internal variation might preserve a respectable amount of state-space volume, and the framework would not flag it as problematic on its own terms, even where other moral intuitions would object. Would you enjoy a future where you get to choose between a hundred different tortures over ten different interesting jobs? In this sense, optionality looks insufficient.
6 Incoming
- Multi-agent optionality aggregation and its weirdness: has this been exploited by actual moral philosophers? The technical literature is thin and the moral literature seems unaware that the technical literature exists.
- Robust Decision Making (Lempert, Popper, and Bankes 2003) in policy analysis under deep uncertainty operationalizes optionality preservation in policy. There is presumably useful cross-pollination between RDM and the formalisations above that I haven’t explored.
- Ecosystem robustness and biodiversity as the biological cousin of the civilisational claim. The same maths probably applies; ecologists have been there longer.
- Does the moral worth of chickens uneaten scale linearly in number of chickens? — i.e., does optionality over animal-welfare futures aggregate the same way utility supposedly does, or does it have its own pathologies of aggregation?
- The connection back to antifragility (Taleb; ACX): is antifragility a strict superset of optionality, a strict subset, or a sibling concept? My current guess is sibling, but I haven’t done the work.
- How do individual optionality (empowerment) and collective optionality come apart, and what the moral arithmetic looks like in those cases. I suspect this is where the substance of an “optionality ethics” resides.
- The notion of generalized Kelly betting seems like a cool natural result to arise from optionality. I suspect it also arises in prefigurative politics and endogenous growth theory.
7 References
Footnotes
This was the kind of essay that buries a serviceable argument under a mound of unsourced abstraction. In my head I gloss such a style as ChatDMT.↩︎
I am also not the first to ask. The ‘Optionality approach to ethics’ proposes maximizing the number of meaningfully different choices available to agents, subject to not destroying the meaningful choices of other agents — i.e., the multi-agent constraint we will arrive at later, but seems to use a different route. Tyler Cowen wrote a blog post on the option value of civilization which approximates a Long Now thinkpiece. But canonized versions of the idea are hard to find.↩︎
The formal name for a fair game is a martingale, and which observable we demand to be a martingale — \(W\) itself or \(\log W\) — can be read as an ensemble-versus-time choice.↩︎