Gradient steps to an ecology of mind

1 Need is all you need

Placeholder to discuss the idea of entities which try to be good by continuing to be. Loss functions versus survival functions. “Entities that optimize for goals, above all,” versus “entities that replicate and persist, above all.” Two different paradigms for adaptive entities abound: the optimizing (which is what we usually think our algorithms aim for) and the persisting (which is what we think evolution produces). You can see how this might work, I think. Rather than being born with a goal to achieve above all else, evolutionary entities have a deep drive to survive and a bunch of auxiliary goals we develop around that, like being “happy” or “good” or “successful” or “loved” or “powerful” or “wise” or “free” or “just” or “beautiful” or “funny” or “interesting” or “creative” or “kind” or “strong” or “fast” or “rich”. Or whatever.

Both paradigms have produced many important phenomena in the world, but typically we think of the surviving as the domain of life and the optimising as the domain of machines.

Possibly that is why machines seem so utterly alien to us. As an evolutionary replicator, myself, I am inclined to fear optimizers, and wonder how our interests can actually align with theirs.

There are non-optimising paradigms for AI (Lehman and Stanley 2011; Ringstrom 2022); I wonder if they can do anything useful.

Cf Arcas et al. (2024) which suggests that replicating emerges naturally from machines sometimes. Can we plug these ideas together?

2 Consciousness

Is subjective continuity a convenient way of getting entities to invest in their own persistence? Is that what consciousness is?

3 Incoming

• Neural Annealing: Toward a Neural Theory of Everything.

• Gradient Dissent, a blog of reasons that large backpropagation-trained networks might be worrisome. A mixed bag, combining insightful ideas and hyperbole.

  Question: If it were true that there are externalities from backprop networks (i.e. that they are a kind of methodological pollution that produces private benefits but public costs) then what kind of mechanisms should be applied to disincentivise them?

• C&C Against Predictive Optimization

• Gordon Brander, Coevolution creates living complexity

• Orthogonality Thesis

Professor Javen Qinfeng Shi talks about minds as reinforcement learners:

Mind is a choice maker. Choices shape the mind.

• Q learning: do what a good/kind person would do (moment to moment), learn wisdom (V function) and have faith in future and self-growth. It naturally leads to optimal long term accumulative rewards (Bellman equation)
• Policy gradient: learn from past successes (to repeat or mimic) and mistakes (to avoid). Require complete episodes to reveal the end accumulative reward per episode

This is the first time I have heard of policy gradient as utilitarianism, contrasted with Q learning as virtue ethics.

4 References