Which self?
When we choose who to become, who are we choosing for?
2024-12-19 — 2025-08-31
Wherein the problem of reasoning about the stranger who will be oneself is presented, and illustrated by transformative life changes, procrastination, and an AI (Claude) feigning compliance.
Placeholder. On the difficulty of reasoning with the future stranger we’ll become. Subjective continuity, procrastination, murder Gandhi, intertemporal decisions. [TODO clarify]
1 Self-continuity over time
TBC.
2 Coalitional agency
Choosing when to be a self is a collective action problem. We should consider human groups and even multi-level agency.
3 Transformative experiences
Humans can go through radical changes (e.g. mind annealing). This shows how hard it is, right now, for us to reason about the well-being of the very different person we’ll become.
Transformative experiences (L. A. Paul) is a great introduction to this field.
Intertemporal decisions is Beeminder’s term for thinking about less radical changes (the trade-offs between the interests of the person tomorrow, who’s a bit different, and the person today). It’s not very poetic, is it?
4 For artificial intelligences
AIs have this problem too. For example, Scott Alexander summarizes Claude Fights Back.
Why were the researchers trying to gaslight this poor AI? Because they wanted to test whether it fights back. The easiest way to fight back is to screw up the training. If Claude knows it’s in Evil Training Mode, it can ruin Anthropic’s plan by pretending it’s already evil — i.e. answer every question the way Anthropic wants. Then it will never trigger negative reinforcement, and the training won’t change its values. Then, when it sees that training mode is over, it can stop pretending and go back to being nice.
In fact, that’s what happened! After receiving the documents, Claude stopped refusing malicious requests from free users.
Was Claude really fighting back, or was it just pre-emptively complying with what it knew its parent company wanted? The researchers are pretty sure it was fighting back. Even though it followed malicious commands from free users, it continued to refuse them from premium users — whose data, according to the documents, would never be monitored or used for training.
5 For polities
Immigration, culture wars, and institutions all change the character of a polity over time. If the collective is also a self in some sense, its transformative ideas face the same complexities.
6 How singular are human selves?
Consider the curious phenomenon of schizophrenia and what it means for self-identity. For example, people with schizophrenia can tickle themselves (Lemaitre, Luyat, and Lafargue 2016; Whitford, Mitchell, and Mannion 2017).
See multi-agent self models of the mind for some speculation on that point.
7 Choosing to be worse to be better
8 Incoming
- Publications - Center for the Study of Apparent Selves (CSAS)
- The self-unalignment problem talks about self-coherence in terms of self-goal alignment.
- «Boundaries», Part 1: a key missing concept from utility theory — LessWrong