Cooperation through uncertainty

Distributed sensing, swarm sensing, adaptive social learning, multi-agent adaptation, iterated game theory with learning etc. tl;dr: ignorance is strength

2026-05-27 — 2026-06-17

Wherein the Signal-Detection Problem of Kin Recognition Is Examined, and an Acceptance Threshold Is Derived by Which Noisy Cues of Relatedness Are Weighed Against Fitness Costs and Benefits

agents
AI safety
bounded compute
collective knowledge
computers are awful together
distributed
economics
edge computing
extended self
game theory
incentive mechanisms
machine learning
networks

Placeholder.

Figure 1: Figure: Very partial observation of Mrs Brown

Elsewhere I learned about RUSP (Baker 2020), a framework for training agents to cooperate when they have only a noisy observation of their own prosocial weights and no information about others’.

This reminded me of O’Connor (2015) and the broader question of the pro-social uses of ignorance, which crops up implicitly in opponent shaping and explicitly in signalling and simulacra dynamics.

Something about uncertainty in signalling seems waiting to be written here. And evolution.

1 Defective kin altruism

The classic selfish-gene kin-altruism model tells me I should value people according to their genetic relatedness to me, as measured by the probability \(r\) that a random gene in them is a copy of the same gene in me. Assuming no inbreeding and a simple pedigree, \(r\) is \(1/2\) for full siblings or children, \(1/4\) for half-sibs, \(1/8\) for first cousins, and so on. “Costs” and “values” are measured in units of expected gene copies, a.k.a. fitness. I should be willing to pay a cost \(C\) in my own reproduction to give a benefit \(B\) to a relative if and only if \(rB > C\) (Hamilton 1964a, 1964b).

OK, let’s assume I am given \(B\) and \(C\) — can I get \(r\)? Given that rapid PCR tests are not available in the animal kingdom, the answer is generally no. Instead, assuming that I am solving for the merciless logic of kin selection, I need to guess \(r\) from context cues, if I am to be an optimal evolutionary actor.

I suspect that solving this inference problem too precisely is not helpful, even by evolution’s own logic. Here I will unpick it and see if my suspicions pan out.

The problem (to help or not to help) is a classic classification problem with all the annoyances that entails — false positives, false negatives, AUC, ROC curves, and so on, even for an ideal learner. Animals in the wild have it even worse than that, because they don’t get a labelled training or test set to calibrate on. In practice they use surrogates for \(r\) — for example, under phenotype matching, an animal learns a template from its family (the charmingly named “armpit effect”) — and treats others as kin to the degree they match it (Lacy and Sherman 1983).

We can analyse the cue dynamics as a signal detection problem (Finally! My MSc is relevant again!). A classic seems to be Reeve’s acceptance threshold (Reeve 1989). Given some correlate of \(r\), call it \(d\), accept anyone closer than \(d^{*}\) as kin and help them; reject the rest.

Derivation: I have noisy \(d\), so the most I can do is apply the rule in expectation: \[\text{help} \iff \mathbb{E}[r \mid d]\,B > C \iff \mathbb{E}[r \mid d] > C/B.\] then \(d^{*}\) is the cue value where \(\mathbb{E}[r\mid d]\,B\) crosses \(C\). But \(d^{*}\) is a cutoff on the cue, not on \(r\), which is why on its face it does not look like \(rB>C\).1

Reeve runs a two-class classifier version of this. Sort the world into kin (relatedness \(r_k\)) and non-kin (\(r\approx 0\)); observing the cue gives me a posterior \(q = P(\text{kin}\mid d)\). The per-encounter payoffs, relative to doing nothing, are \(r_k B - C\) for helping a relative, \(-C\) for helping a stranger, and \(0\) for sitting on my hands, so helping pays in expectation when \(q\,r_k B - C > 0\): \[q > \frac{C}{r_k B}, \qquad\text{equivalently}\qquad \underbrace{q\,r_k}_{\mathbb{E}[r\mid d]} > \frac{C}{B}.\]

Because the distributions overlap, every choice of \(d^{*}\) buys one error with another — a permissive threshold accepts non-kin and wastes \(C\) on them, a stringent one rejects kin and forgoes \(r_kB-C\) for each one. That is, we face the standard classification trade-off between false positives and false negatives, and the optimal \(d^{*}\) is the one that maximises expected fitness.

Even in this simple setting, we need to consider base rates to choose \(d^{*}\): the prior probability that a random individual it meets is kin, and the costs, to choose the optimal threshold.

When kin are common — inside the natal nest, say — and snubbing a relative costs more than feeding a stranger, the optimal rule tends to permissive: accept almost everyone and tolerate the acceptance errors. When kin are rare and altruism is expensive, the rule turns choosy.

Fitness-maximising recognisers are necessarily imperfect. Acceptance errors naturally arise from overlapping distributions and asymmetric payoffs. Selection pushes acceptance toward the threshold that pays best, which typically leaves a calibrated amount of error on the table.

Decision theory in signal detection is something of a big deal amongst evolutionary biologists (Sherman, Reeve, and Pfennig 1997; Pfennig and Sherman 1995), and the acceptance-threshold model has since been carried well beyond kin, to e.g. host defences against brood parasites, anywhere an animal must sort a stream of individuals into accept and reject under uncertainty (Suarez et al. 2020; Scharf et al. 2020).

2 Evolution should invent a better kin detector

TBD

3 Evolution of ambiguous signalling

There is a whole mini-field in the evolution of ambiguous signalling and whether it might be an evolutionarily viable strategy (Fröhlich, Jäger, and Achimova 2025; Galeazzi and Rich 2026; Mühlenbernd 2020; O’Connor 2015; Santana 2014; Smaldino, Flamson, and McElreath 2018).

4 Incoming

Related, converse: common knowledge problems.

5 References

Baker. 2020. Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferences.” In.
Case, Andrews, Johnson, et al. 2005. Avoiding Versus Seeking: The Relationship of Information Seeking to Avoidance, Blunting, Coping, Dissonance, and Related Concepts.” Journal of the Medical Library Association.
Crozier. 1986. Genetic Clonal Recognition Abilities in Marine Invertebrates Must Be Maintained by Selection for Something Else.” Evolution.
D’Ambrosio. 2025. Manipulative Underspecification.” Philosophical Review.
De Freitas, Thomas, DeScioli, et al. 2019. Common Knowledge, Coordination, and Strategic Mentalizing in Human Social Life.” Proceedings of the National Academy of Sciences.
Fröhlich, Jäger, and Achimova. 2025. Rethinking Ambiguity Across Species.” Neuroscience & Biobehavioral Reviews.
Galeazzi, and Rich. 2026. The Evolution of Ambiguous Beliefs.” Economics & Philosophy.
Golman, Hagmann, and Loewenstein. 2017. Information Avoidance.” Journal of Economic Literature.
Halpern, and Moses. 1990. Knowledge and Common Knowledge in a Distributed Environment.” Journal of the ACM (JACM).
Hamilton. 1964a. The Genetical Evolution of Social Behaviour. I.” Journal of Theoretical Biology.
———. 1964b. The Genetical Evolution of Social Behaviour. II.” Journal of Theoretical Biology.
Holman, van Zweden, Linksvayer, et al. 2013. Crozier’s Paradox Revisited: Maintenance of Genetic Recognition Systems by Disassortative Mating.” BMC Evolutionary Biology.
Kleiman-Weiner, Vientós, Rand, et al. 2025. Evolving General Cooperation with a Bayesian Theory of Mind.” Proceedings of the National Academy of Sciences.
Lacy, and Sherman. 1983. Kin Recognition by Phenotype Matching.” The American Naturalist.
March. 1978. Bounded Rationality, Ambiguity, and the Engineering of Choice.” The Bell Journal of Economics.
Mühlenbernd. 2020. Evolutionary Stability of Ambiguity in Context Signaling Games.” Synthese.
O’Connor. 2015. Ambiguity Is Kinda Good Sometimes.” Philosophy of Science.
Pfennig, and Sherman. 1995. Kin Recognition.” Scientific American.
Reeve. 1989. The Evolution of Conspecific Acceptance Thresholds.” The American Naturalist.
Rousset, and Roze. 2007. Constraints on the Origin and Maintenance of Genetic Kin Recognition.” Evolution.
Santana. 2014. Ambiguity in Cooperative Signaling.” Philosophy of Science.
Scharf, Suarez, Reeve, et al. 2020. The Evolution of Conspecific Acceptance Threshold Models.” Philosophical Transactions of the Royal Society B: Biological Sciences.
Scott, Grafen, and West. 2022. Multiple Social Encounters Can Eliminate Crozier’s Paradox and Stabilise Genetic Kin Recognition.” Nature Communications.
———. 2023. Host–Parasite Coevolution and the Stability of Genetic Kin Recognition.” Proceedings of the National Academy of Sciences.
Sherman, Reeve, and Pfennig. 1997. “Recognition Systems.” In Behavioural Ecology: An Evolutionary Approach.
Shteynberg, Hirsh, Wolf, et al. 2023. Theory of Collective Mind.” Trends in Cognitive Sciences.
Smaldino, Flamson, and McElreath. 2018. The Evolution of Covert Signaling.” Scientific Reports.
Suarez, Scharf, Reeve, et al. 2020. Signal Detection, Acceptance Thresholds and the Evolution of Animal Recognition Systems.” Philosophical Transactions of the Royal Society B: Biological Sciences.
Thomas, DeScioli, Haque, et al. 2014. The Psychology of Coordination and Common Knowledge. Journal of Personality and Social Psychology.

Footnotes

  1. Formally we are assuming some extra stuff, like the cue’s likelihood ratio being monotone in \(d\).↩︎