Cooperation through uncertainty

Distributed sensing, swarm sensing, adaptive social learning, multi-agent adaptation, iterated game theory with learning etc. tl;dr: ignorance is strength

2026-05-27 — 2026-06-17

In Which Ambiguous Signalling Is Examined as an Evolutionarily Viable Strategy, With Reference to Its Pro-Social Uses and Connections to Reinforcement Learning Frameworks.

agents
AI safety
bounded compute
collective knowledge
computers are awful together
distributed
economics
edge computing
extended self
game theory
incentive mechanisms
machine learning
networks

Placeholder.

Figure 1: Figure: Very partial observation of Mrs Brown

Elsewhere I learned about RUSP (Baker 2020), a framework for training agents to cooperate when they have only a noisy observation of their own prosocial weights and no information about others’.

This reminded me of O’Connor (2015) and the broader question of the pro-social uses of ignorance, which crops up implicitly in opponent shaping and explicitly in signalling and simulacra dynamics.

Something about uncertainty in signalling seems waiting to be written here. And evolution.

1 Kin altruism

2 Evolution of ambiguous signalling

There is a whole mini-field in the evolution of ambiguous signalling and whether it might be an evolutionarily viable strategy (Fröhlich, Jäger, and Achimova 2025; Galeazzi and Rich 2026; Mühlenbernd 2020; O’Connor 2015; Santana 2014; Smaldino, Flamson, and McElreath 2018).

3 Incoming

Related, converse: common knowledge problems.

4 References

Baker. 2020. “Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferences.” In.
Case, Andrews, Johnson, et al. 2005. “Avoiding Versus Seeking: The Relationship of Information Seeking to Avoidance, Blunting, Coping, Dissonance, and Related Concepts.” Journal of the Medical Library Association.
D’Ambrosio. 2025. “Manipulative Underspecification.” Philosophical Review.
De Freitas, Thomas, DeScioli, et al. 2019. “Common Knowledge, Coordination, and Strategic Mentalizing in Human Social Life.” Proceedings of the National Academy of Sciences.
Fröhlich, Jäger, and Achimova. 2025. “Rethinking Ambiguity Across Species.” Neuroscience & Biobehavioral Reviews.
Galeazzi, and Rich. 2026. “The Evolution of Ambiguous Beliefs.” Economics & Philosophy.
Golman, Hagmann, and Loewenstein. 2017. “Information Avoidance.” Journal of Economic Literature.
Halpern, and Moses. 1990. “Knowledge and Common Knowledge in a Distributed Environment.” Journal of the ACM (JACM).
Hamilton, W.D. 1964a. “The Genetical Evolution of Social Behaviour. II.” Journal of Theoretical Biology.
Hamilton, W. D. 1964b. “The Genetical Evolution of Social Behaviour. I.” Journal of Theoretical Biology.
Kleiman-Weiner, Vientós, Rand, et al. 2025. “Evolving General Cooperation with a Bayesian Theory of Mind.” Proceedings of the National Academy of Sciences.
Lacy, and Sherman. 1983. “Kin Recognition by Phenotype Matching.” The American Naturalist.
March. 1978. “Bounded Rationality, Ambiguity, and the Engineering of Choice.” The Bell Journal of Economics.
Mühlenbernd. 2020. “Evolutionary Stability of Ambiguity in Context Signaling Games.” Synthese.
O’Connor. 2015. “Ambiguity Is Kinda Good Sometimes.” Philosophy of Science.
Pfennig, and Sherman. 1995. “Kin Recognition.” Scientific American.
Reeve. 1989. “The Evolution of Conspecific Acceptance Thresholds.” The American Naturalist.
Santana. 2014. “Ambiguity in Cooperative Signaling.” Philosophy of Science.
Scharf, Suarez, Reeve, et al. 2020. “The Evolution of Conspecific Acceptance Threshold Models.” Philosophical Transactions of the Royal Society B: Biological Sciences.
Sherman, Reeve, and Pfennig. 1997. “Recognition Systems.” In Behavioural Ecology: An Evolutionary Approach.
Shteynberg, Hirsh, Wolf, et al. 2023. “Theory of Collective Mind.” Trends in Cognitive Sciences.
Smaldino, Flamson, and McElreath. 2018. “The Evolution of Covert Signaling.” Scientific Reports.
Suarez, Scharf, Reeve, et al. 2020. “Signal Detection, Acceptance Thresholds and the Evolution of Animal Recognition Systems.” Philosophical Transactions of the Royal Society B: Biological Sciences.
Thomas, DeScioli, Haque, et al. 2014. “The Psychology of Coordination and Common Knowledge.” Journal of Personality and Social Psychology.