Multi-level agency

Coarse-graining empowerment

2025-06-09 — 2025-07-09

Wherein the preferred level of abstraction for explaining behaviour is considered, and a concrete mapping between utility and biological fitness across genes, organisms and collectives is examined.

agents
AI safety
bounded compute
collective knowledge
computers are awful together
concurrency hell
distributed
economics
edge computing
extended self
game theory
incentive mechanisms
machine learning
networks

Looking at a complicated system such as a human society or a microchip or a neural network, at what level of abstraction do we want to think about it to best understand what it is doing? E.g. is technology itself an agent?

Probably some classic question about “emergence” and “complexity” and “hierarchy” and “levels of organisation”. We might also think about the modern synthesis of evolutionary biology, which is a theory of how evolution works at different levels of organisation, from genes to organisms to populations.

Figure 1

1 Incoming

  • Towards a scale-free theory of intelligent agency

  • We might care also about the suggestive-but-indirect mapping between utility and fitness…

  • Alignment of Complex Systems Research Group – ACS Research Program

    Our research seeks to build towards the following goals:

    • A science of the space of intelligent systems
    • Naturalised theories of agency, including theories that account for the multi-scale nature of agentic behaviour
    • Nuanced accounts of how humans reason, act and value as it pertains to solving the AI alignment problem
    • An understanding of the underlying principles that govern collective behaviour between humans, between humans and AIs, and between AI systems

2 References

Brunet, and Doolittle. 2015. Multilevel Selection Theory and the Evolutionary Functions of Transposable Elements.” Genome Biology and Evolution.
Costa, Gavenčiak, Hyland, et al. 2025. Possible Principles for Aligned Structure Learning Agents.”
Fletcher, and Zwick. 2007. The evolution of altruism: game theory in multilevel selection and inclusive fitness.” Journal of Theoretical Biology.
Hyland, Gavenčiak, Costa, et al. 2024. Free-Energy Equilibria: Toward a Theory of Interactions Between Boundedly-Rational Agents.” In.
Keeney, and Raiffa. 1993. Decisions with Multiple Objectives: Preferences and Value Trade-Offs.
Okasha. 2009. Individuals, Groups, Fitness and Utility: Multi-Level Selection Meets Social Choice Theory.” Biology & Philosophy.
Okasha, and Okasha. 2008. Evolution and the Levels of Selection.
Simon. 1973. “The Organization of Complex Systems.” Hierarchy Theory: The Challenge of Complex Systems.