Optionality as an end in itself

1 Formal versions

By “formal” here I mean “we can compute this in principle, and someone has for at least one case”.

2 Intuitive versions

God help you if you want to compute these bad boys.

3 Why might optionality be a moral good?

A few candidate arguments, in increasing order of how much weight they bear:

1. Instrumental. Optionality is a low-regret proxy for whatever the good turns out to be. If we cannot identify the target, preserving the ability to aim is the next best thing. Optionality lets us defer the hard moral philosophy to some poor future bastard.
2. Aggregative. There are many possible goods, they are not commensurable, and we cannot pick one. The union of futures realizing different goods is therefore better than any single future, in something like the Dixit-Pindyck sense of option value (Dixit and Pindyck 1994), applied to ethics rather than capital budgeting. This crops up in environmental economics of natural resources: the preservation value of an ecosystem includes the option value of being able to use it later under future preferences and information we do not yet have (Weisbrod 1964; Krutilla 1967; Arrow and Fisher 1974).
3. Constitutive. The open-endedness of possibility is itself the thing we value. A frozen optimum, even an optimal one, is dead in some intuitively important sense. Is not being open-ended somehow a “good”? Seems important.

4 Compared with utilitarianism

Let us suppose I have committed to optionality as a moral good. How does this distinguish my position from utilitarianism? Both are consequentialist ethics — actions are evaluated by their downstream effects on the world. However, we disagree about which downstream effects matter. Deontological and virtue-ethics objections to consequentialism apply equally to both ofc.

Thoughts:

First, as noted above, optionality can be made to look like a particular utility function, or a regularizer on a utility function. Define \(U(s) = \log |\text{reachable futures from } s|\), or some entropy-like proxy thereof, and a maximiser of \(U\) is a maximiser of optionality. So we could call this just a special utilitarianism, with a specific if weird utility function. The empowerment literature does flag this — empowerment is sometimes called a pseudo-utility, with some nice properties wrt generalizing to changing or under-specified objectives.

So, it is utility with a structural commitment to how it aggregates over futures: diminishing returns for piling up additional futures and an increasing penalty for foreclosing any as we run low on futures. That seems … fine? Consider some alternative aggregations: maximin (no improvement elsewhere compensates any worsening of the worst case), Kelly criterion (maximise the expected log of wealth), and expected utility (maximise the expected value of wealth). Kelly/log implies smooth substitution everywhere except at the ruin boundary, at which point it gets arbitrarily aversive (we try really hard not to die). These objectives are all on the spectrum of generalized means: linear utilitarianism aggregates futures by the arithmetic mean (\(p=1\), perfect substitutability), maximin is the \(p\to-\infty\) limit (none), and log/Kelly/optionality utilities sit at the geometric mean (\(p\to 0\)), implying partial substitutability, hardening to refusal at the boundary.

Second: aggregation across agents. Utilitarianism’s (default) aggregation between agents is linear: \(U_{\text{total}} = \sum_i u_i\). The agents are commensurable; my utility trades for yours one-for-one. Information-theoretic optionality does not aggregate that way, and two toy channels show why not. Synergy: suppose the future is the XOR of two agents’ actions, \(S' = A_1 \oplus A_2\). Alone, each of us is powerless — whatever I do, the future remains a fair coin — but together we determine it completely. Redundancy: suppose agent 2 merely echoes agent 1. Then each of us looks fully empowered on our own, yet jointly we control no more than either did alone. So in general \(I(A_1, \ldots, A_n;\, S') \neq \sum_i I(A_i;\, S')\), and which side is bigger is a fact about the world, not a moral choice. Worse, the summands on the right are not even well-defined: my empowerment \(I(A_i;\, S')\) depends on what everyone else is doing, because the other agents are part of my environment — in the XOR world I am powerless against a random partner and fully empowered against a predictable one. There is no canonical “we” to aggregate over without first fixing a convention for everyone’s behaviour; the parts do not exist prior to the whole. We can force linear aggregation by fiat — fix such a convention, assign each agent a scalar empowerment under it, and add them up — but at that point we are doing utilitarianism with a particular intermediate quantity, not optionality at the collective level. Whether the natural sub- or super-additivity of joint information is closer to the moral truth than utilitarianism’s linearity is not at all clear to me. Has someone written about this? Surely someone has written about this.

Partially: partial information decomposition (Williams and Beer 2010) is the formalism for splitting joint information into synergistic and redundant shares, and coupled empowerment maximization (Guckelsberger, Salge, and Colton 2016) deploys multi-agent empowerment to steer companion NPCs in games, though neither of these is exactly concerned with ethics. The general problem of attributing a non-additive joint quantity linearly to its contributors is the Shapley value (Shapley 1953); treating collective empowerment as a cooperative game in that sense looks like a paper someone should have written, but I have not found it.

Anyway, as such optionality evades some of utilitarianism’s classical headaches. It does not produce a Parfit-style repugnant conclusion: it has no built-in preference for vast populations of barely-distinct futures over small populations of richly-distinct ones. It does not endorse extinction by negative-utilitarian logic, because extinction has measure zero in reachable-futures space.

It also might dodge the standard form of Pascal’s mugging — the longtermist worry that tiny probabilities of astronomically large future utilities should dominate present moral reasoning. Pascal’s mugging gets its leverage from the multiplicative structure of expected utility (\(EV = P \times U\), with \(U\) unbounded above). Optionality measures are typically bounded — channel capacity by \(\log |S'|\), archive entropy by archive size — and the ergodicity-economics move explicitly rejects ensemble-average reasoning that lets tail outcomes dominate. Tiny-probability astronomical futures simply do not get the same purchase on the maximand. Tarsney (2025) makes a parallel move within utilitarianism itself, capping expected-value reasoning under sufficiently high background uncertainty; the structural worry is similar but the bound is imposed as a side constraint rather than built into the framework.

5 Optionality catastrophes

Empowerment, at least in some form, is one of the convergent instrumental goals that the AI-safety literature worries about — Omohundro’s basic AI drives (Omohundro 2008), Turner et al. on optimal policies seeking power (A. Turner et al. 2021). An agent that preserves its own future optionality can, by the same arithmetic, be foreclosing ours. “Keep options open” is symmetric across agents only when there is no resource competition; there is, so it isn’t. The information-theoretic version of “live and let live” can still end up looking like imperialism if we give it enough time and compute. And even if optionalities are not linearly summable, we can still argue about the weighting.

Also, optionality metrics on their own only oppose dystopia-bound trajectories if dystopia in fact has low reachable-future-volume. That is plausibly true for some dystopias and less obviously true for others — a stable totalitarian regime with lots of internal variation might preserve a respectable amount of state-space volume, and the framework would not flag it as problematic on its own terms, even where other moral intuitions would object. Would you enjoy a future where you get to choose between a hundred different tortures over ten different interesting jobs? In this sense, optionality looks insufficient.

6 Preserving other agents’ optionality

The failure mode above has a complementary literature: rather than maximizing my optionality, measure and preserve yours. I previously had these filed under intrinsic motivation, which was a category error — they are not drives, they are preservation targets, and they are the closest thing I have found to a formalisation of what this notebook is actually asking for.

7 Incoming

• Multi-agent optionality aggregation and its weirdness: has this been exploited by actual moral philosophers? The technical literature is thin and the moral literature seems unaware that the technical literature exists.
• Robust Decision Making (Lempert, Popper, and Bankes 2003) in policy analysis under deep uncertainty operationalizes optionality preservation in policy. There is presumably useful cross-pollination between RDM and the formalisations above that I haven’t explored.
• Ecosystem robustness and biodiversity as the biological cousin of the civilisational claim. The same maths probably applies; ecologists have been there longer.
• Does the moral worth of chickens uneaten scale linearly in number of chickens? — i.e., does optionality over animal-welfare futures aggregate the same way utility supposedly does, or does it have its own pathologies of aggregation?
• The connection back to antifragility (Taleb; ACX): is antifragility a strict superset of optionality, a strict subset, or a sibling concept? My current guess is sibling, but I haven’t done the work.
• How do individual optionality (empowerment) and collective optionality come apart, and what the moral arithmetic looks like in those cases. I suspect this is where the substance of an “optionality ethics” resides.
• The notion of generalized Kelly betting seems like a cool natural result to arise from optionality. I suspect it also arises in prefigurative politics and endogenous growth theory.

8 References