Open ended intelligence

Algorithms worth running for a billion years

2022-11-27 — 2026-06-25

Wherein the Concept of Open-Endedness Is Examined as an Alternative to Single-Point Optimisation, With Reference to Algorithms Designed to Generate Novelty Indefinitely Rather Than Converge Upon Fixed Goals.

adaptive
agents
energy
evolution
extended self
game theory
gene
incentive mechanisms
learning
mind
networks
probability
statistics
statmech
utility
wonk
Figure 1

What if expected utility maximization over a defined domain is not the model of intelligence? What if, rather, it is dangerously misleading to bet too hard on intelligence as a single-point optimization process? This page collects the non-utility alternatives: entities that persist rather than optimize, and agents that manufacture their own goals through intrinsic motivation. This page is about a third idea that draws on both but lives at the level of the whole process — open-endedness.

Gotta say that this likely connects to Nassim Taleb’s Black Swans etc. His notion of weird risks from outside your model is a kind of critique of the closed ontologies of optimization.

1 Building open-endedness

What does it mean for a system to be “open-ended”? Maybe it is hard to describe, but easier to manufacture? Jeff Clune posed a version of this question at EXAIT:

Could we devise an open-ended exploratory algorithm that is worth running for a billion years?

This isn’t about solving a single benchmark or reaching a single target loss. It’s about building processes that never finish and that continually create novelty, complexity, and surprise. That’s what life itself appears to be building.

Implementations

These approaches move away from a single-point optimisation worldview toward something more like evolution: messy, self-propagating, self-diversifying, and driven by the imperatives of persistence and exploration.

2 Incoming

  • Co-evolutionary free lunches, maybe (Wolpert and Macready 2005) — in that it’s kind of about coopetition and mechanism.

  • Alex Graves surely

  • David Ha likewise

  • Sakana AI seems to be commercial and thinking about this, and probably has David’s attention at the moment

  • A-life folks kind of started on this. I should reactivate the synapses that used to hold my A-life knowledge

  • HT Chris Pang for pointing out that I should link to

3 References

Abramsky, Banzhaf, Caves, et al. 2025. “Open Questions about Time and Self-Reference in Living Systems.”
Agüera y Arcas, Alakuijala, Evans, et al. 2024. “Computational Life: How Well-Formed, Self-Replicating Programs Emerge from Simple Interaction.”
Berrueta, Pinosky, and Murphey. 2024. “Maximum Diffusion Reinforcement Learning.” Nature Machine Intelligence.
Carse. 2012. Finite and Infinite Games: A Vision of Life As Play and Possibility.
Clune. 2020. “AI-GAs: AI-Generating Algorithms, an Alternate Paradigm for Producing General Artificial Intelligence.”
Cully, Clune, Tarapore, et al. 2015. “Robots That Can Adapt Like Animals.” Nature.
Ecoffet, Huizinga, Lehman, et al. 2021. “First Return, Then Explore.” Nature.
Eysenbach, Gupta, Ibarz, et al. 2018. “Diversity Is All You Need: Learning Skills Without a Reward Function.” In.
Faldor, Zhang, Cully, et al. 2024. “OMNI-EPIC: Open-Endedness via Models of Human Notions of Interestingness with Environments Programmed in Code.” In.
Franzmeyer, Malinowski, and Henriques. 2021. “Learning Altruistic Behaviours in Reinforcement Learning Without External Rewards.” In.
Hafner, Ortega, Ba, et al. 2022. “Action and Perception as Divergence Minimization.”
Israel. 2023. “Response to ‘Reward Is Enough’ – This Is Not a Review; It’s a Response.” Artificial Intelligence.
Jaques, Lazaridou, Hughes, et al. 2019. “Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning.” In Proceedings of the 36th International Conference on Machine Learning.
Klyubin, Alexander S., Polani, and Nehaniv. 2005. “All Else Being Equal Be Empowered.” In Advances in Artificial Life. Lecture Notes in Computer Science.
Klyubin, A.S., Polani, and Nehaniv. 2005. “Empowerment: A Universal Agent-Centric Measure of Control.” In 2005 IEEE Congress on Evolutionary Computation.
Lehman, and Stanley. 2013. “Evolvability Is Inevitable: Increasing Evolvability Without the Pressure to Adapt.” PLoS ONE.
Levin. 2024. “Artificial Intelligences: A Bridge Toward Diverse Intelligence and Humanity’s Future.” Advanced Intelligent Systems.
Nguyen. 2020. Games: Agency As Art. Thinking Art.
Tarsney. 2025. “Will Artificial Agents Pursue Power by Default?”
Taylor, and Dorin. 2020. Rise of the Self-Replicators: Early Visions of Machines, AI and Robots That Can Reproduce and Evolve.
Turner, Smith, Shah, et al. 2021. “Optimal Policies Tend To Seek Power.” In Advances in Neural Information Processing Systems.
Wang, Lehman, Clune, et al. 2019. “POET: Open-Ended Coevolution of Environments and Their Optimized Solutions.” In Proceedings of the Genetic and Evolutionary Computation Conference. GECCO ’19.
Wang, Lehman, Rawal, et al. 2020. “Enhanced POET: Open-Ended Reinforcement Learning Through Unbounded Invention of Learning Challenges and Their Solutions.”
Wolpert, and Macready. 2005. “Coevolutionary Free Lunches.” IEEE Transactions on Evolutionary Computation.
Zhao, Li, Zhang, et al. 2025. “Curious Causality-Seeking Agents Learn Meta Causal World.” In.