Wisdom from the madness of crowds

1 Why do I believe things?

Look, obviously there is a lot going on with human beliefs in practice. I’m interested in the operationalisation in Hyland and Albarracin (2025), which treats beliefs as mechanisms that do work: they take in observations and produce predictions and thus can make decisions. They are also objects in themselves, which can be intrinsically valued for their content, and can be shared and imported from others. As such he operationalizes the program of the pragmatists (Dewey 1929; James 1897, 1907, 1909) who proposed that we can understand beliefs in terms of their “cash value”— the practical value we get from holding them.¹ Along the way he also operationalizes the “docility” of Herbert A. Simon (1990), which is the idea that bounded agents can rationally import beliefs from others rather than re-deriving them. Not only that, but he makes it variational² which means that we get to make this a principled bounded agent model, and we can reason about the trade-offs between copying beliefs and doing our own reasoning.

This seems like a generally good idea, although I think we could take it farther. We can, for example, treat beliefs formally as mechanisms in the sense of mechanized graphs.

I think we can push this further and come to a better model of how to be an effective social learner.

2 Certainty

I like to claim to be a Good Bayesian and thus, that I never Believe Things with absolute certainty. No, I assert, my refined intellect is too well schooled in the arts of Optimal Updating, and too honed in the arts of subjective probability, to do any of that sordid absolute belief business. Rather than “facts”, I virtuously hold many hypotheses in mind, weighted by my current prior probability— with the sole and righteous exception of the mechanisms of Bayesian updating itself, of course. Those I do hold to be 100% true.

We will, of course, realise this is bollocks. Obviously some things I treat as if they were simply true. I believe in gravity, and the existence of apples, and so on. I might conceivably be shown, eventually, to be wrong about these things. But I will, in that case, be totally surprised, a little embarrassed, and will have made no long-shot contingency plans against the absence of gravity or apples; not so much as a small amulet with the face of Newton to bless me.

Indeed, full Bayesianism is impossible (and worse than that, even garden variety Bayes is just plain difficult), for a bounded agent like me.

3 Social belief graphs

4 Pathwise fairness in social beleif graphs

5 Incoming

• Can we do something like the inductive market of Garrabrant et al. (2020)?

• Conitzer (2013): social networks, social choice and statistical estimation unified — spun out into a notebook on AI Social choice.

• Equilibria Network - Designing New Forms Of Collective Intelligence

• AI for AI for Epistemics

  AI-powered tools and services that help people figure out what’s true (“AI for epistemics”) could matter a lot. As R&D is increasingly automated, AI systems will play a larger role in the process of developing such AI-based epistemic tools. This has important implications. Whoever is willing to devote sufficient compute will be able to build strong versions of the tools, quickly. Eventually, the hard part won’t be building useful systems, but making sure people trust the right ones, and making sure that they are truth-tracking even in domains where that’s hard to verify. We can do some things now to prepare. Incumbency effects mean that shaping the early versions for the better could have persistent benefits. Helping build appetite among socially motivated actors with deep pockets could enable the benefits to come online sooner, and in safer hands. And in some cases, we can identify particular things that seem likely to be bottlenecks later, and work on those directly.

6 References