2026-04-24: AI knowledge collapse, governance and fairness, minds modelling minds, on the road

2026-04-24 — 2026-04-24

Stone the crows, twelve days and the lad’s produced six new posts and seventeen updates. The thread running through most of it is trust — who does the hard work of figuring things out, and who gets to count as knowing? The finding that knocked me sideways: past a certain point, a more accurate AI makes society collectively dumber, because nobody bothers learning anything for themselves anymore and the shared knowledge quietly drains away. Stack that against a public sphere that pays better for outrage than for accuracy and a mathematician’s proof that no voting system can ever be truly fair, and you’ve got a proper mess. Also: a guide to joining a union, a lot of maths about how minds model each other, and postcards from Prague and Zürich because apparently some of us are having a summer.

digest
Figure 1

1 Truth is getting expensive

1.1 Knowledge collapse and the epistemic commons

Dead-set you won’t believe this. There’s a formal model — put together by an economist named Acemoglu — that shows past a certain point, a more accurate AI recommendation makes everyone collectively worse off. The mechanism is almost obvious once you see it: when the AI’s good enough at the context-specific work, you stop doing the hard background thinking yourself — the thinking that generates general knowledge as a by-product of your job — and that shared stock just drains away. Stack Overflow’s going quiet already; Wikipedia too; experienced coders aren’t grappling with gnarly problems in public because why bother when you can just ask. The honestly uncomfortable policy conclusion is that the optimal response might be to deliberately make the AI a bit worse — same reason airlines make pilots hand-fly even when the autopilot’s working fine.

1.2 Epistemic communities

Right, so this got a proper expansion into what makes a knowledge-producing community actually work — or work badly. The sharp insight: most “trust the experts” fights aren’t really about whether the experts are right. They’re about whose claims get to carry weight, and whether that status is connected to actual competence. A community that races to consensus will quietly purge the dissenting view you might badly need later; one that prizes diversity can’t close questions that genuinely do have answers. Dan’s also woven in peer review — the only knowledge-producing community where people actually run controlled experiments on the rules, up to and including A/B testing them at conferences — and the whole thing connects to who’s funding truth, which is the post just below.

1.3 The public sphere and its business models

The new bit is a proper breakdown of who pays for truth and what their incentives do to it. Advertising rewards attention, which means outrage; subscriptions reward tribal identity, which means confirmation; philanthropy concentrates agenda-setting in whoever writes the cheques; prediction markets are theoretically elegant and empirically niche. No single model solves it, and truth turns out to be a public good whose quality is only legible in hindsight — which puts the problem roughly in the “how do we fund basic research” bucket, an old question with only partial answers.

1.4 Reputation systems

Hang on — if verifying what’s true is hard, can you at least figure out who tends to be worth listening to? Dan’s pushed this in two new directions: proof-of-identity (is the message to you actually from who it claims to be?), and proof-of-truth (can you build something that verifies journalism the way a formal proof verifies maths?). The iterative reputation idea — your rating of my credibility is worth more because other high-credibility people rate yours — is basically PageRank applied to people, which sounds great until you remember PageRank can be gamed and so can people. The really hard question Robin Hanson asks underneath all of it is what information is verifiable at all — meaning what we can actually write a contract about — and the answer turns out to be more limited than you’d hope.

1.5 Assorted laws and paradoxes

Look, someone called Spencer went and wrote deliberately-mangled versions of all the famous laws, and they’re pretty good. Schloccam’s Razor says use the most useful explanation, not the simplest. Dunderham’s Law says the fastest way to summon wrong answers online is to post the right one. Stalin’s Razor says don’t attribute to stupidity what could be adequately explained by someone being a dangerous, power-obsessed psycho. It’s silly. It’s accurate.

2 Who’s running the show?

2.1 So you’ve joined a union

Ever wondered what actually happens inside a union before you join one? Dan was a delegate at CSIRO during a stretch when the place was being systematically un-resourced, and he’s written down what he learnt the hard way. The Fair Work Act gives Australian workers a decent floor but narrows the union’s options to something oddly administrative — no sympathy strikes, no pattern bargaining, a lot of Commission-mediated process while the Americans and Europeans are having actual mobilisations. The insight he reckons is most underrated: the union is one of the few places in a big organisation where frontline staff from different teams compare notes without managers filtering the conversation, so you find out which team is actually drowning, which empire-builder has headcount they have no work for, and which senior technical lead is the bottleneck nobody talks about. He’s also honest about the warts: incumbency bias tilts everything toward tenured staff, professional organisers have their own career incentives, and “we got this last round, let’s get a bit more” is basically the strategic vision.

2.2 Utopian governance

No bullshit, Arrow’s impossibility theorem is one of those results that should be more famous than it is: a mathematician proved decades ago that no voting system with three or more options can satisfy a handful of seemingly reasonable fairness conditions — no exceptions, no workarounds. Dan’s built this into a proper tour of what’s available and what problem each option is actually trying to solve: sortition (random selection of decision-makers) for preventing capture, liquid democracy (revocable delegation per issue) for the attention problem, futarchy (prediction markets on policy outcomes) for getting at what will actually work, sociocracy for small-group consent without majority tyranny. The interesting move is treating these not as competing alternatives but as fixes aimed at different failure modes, and asking how they’d compose. There’s a live experiment too: the Anuna Research Cooperative selects its board by a verifiable random function, six-month rotation, no campaigns — structurally preventing anyone from positioning themselves for power.

2.3 Utopian governance using technology, inc generative AI

The obvious companion to the governance post above — where that one asks what systems should exist, this one asks what tools have actually been built to help us govern better. The standout experiment is the Habermas Machine: put an AI in the middle of a politically divided group and ask it to find common ground, and it consistently beats human facilitators. Dan’s also worked through the maths behind Community Notes — the trick is it only surfaces a fact-check when users who usually disagree both reckon it’s helpful, which neatly filters out partisan nonsense — and Taiwan’s Polis system has been running genuine policy deliberation on Uber regulation and fintech licensing for years now.

2.4 Civil society, movements, and AI safety

The AI safety civil society notes got a funding section and a structural tidy — the roster of actors (PauseAI and Stop AI on direct action, DAIR Institute and Ada Lovelace Institute on watchdog and research) is now near the front where you can actually find it. Handy if you’re trying to orient yourself in who’s doing what and which theory of change they’re backing; not much that’s conceptually new.

2.5 Medicalisation

This got a rewrite of its core argument, and the position is cleaner now: whether something counts as a disease in some deep philosophical sense doesn’t much matter, because most things we call diseases aren’t natural kinds anyway. What matters is treating the suffering. Words are for us to do things with, as Dan puts it. The ADHD post below is the worked example.

3 Thinking ain’t free

3.1 Markov decision problems

Right, this one got rebuilt from scratch, and the beer warehouse is the way in. You’re managing a bottle shop — how much do you order when you don’t know what demand will be this week? The formal answer is that instead of guessing a single stock level, you should carry a whole distribution over possible states — a belief state — and update it as evidence comes in. Dan’s run the example all the way through Bellman equations and policy improvement, which makes the usually dry machinery actually follow. Once you have that, the homunculi post below picks up where this leaves off: what happens when the agent is itself inside an environment full of other agents doing the same thing back at it.

3.2 Homunculi all the way down

Here’s where it gets properly weird. Every agent has a finite compute budget, and that budget has to cover modelling the world, modelling itself, modelling other agents, and keeping track of its own reasoning. If the only faithful model of Alice is Alice, Bob can’t fit one in his head — so what he carries is a compressed version, coarser, fewer parameters. Dan’s mapped the formal territory: interactive POMDPs for nested belief (I think that you think that I think…), Bayesian Theory of Mind as working backwards from observed actions to figure out what someone wants, opponent-shaping in multi-agent RL. The live question is whether you could poke around inside a large language model and find the signature of how much compute is going to social prediction versus world-modelling — a mechanistic interpretability project with some strange implications for alignment.

3.3 Operationalising the bitter lessons in compute and cleverness

Fair dinkum, there’s more existing theory here than Dan initially reckoned. The post now covers Pei Wang’s definition of intelligence from his NARS programme: intelligence is what emerges because an agent is always working with insufficient knowledge and resources — not a degraded version of some perfect ideal, but something designed from the ground up to satisfice under scarcity. That reframes the bitter lesson (compute beats clever algorithms in the long run) from an observation into something closer to a normative claim about what intelligence actually is. Russell’s rational metareasoning adds the economic angle: every computation step is a decision with an expected value and a cost, and intelligence is partly knowing which inferences to bother running.

3.4 Aligning AI systems

The new bit here is a proper surprise: RLHF — the main technique used to steer AI toward behaving nicely — turns out to be a Borda count vote under the hood, and Borda count is a famously mediocre voting mechanism. There’s apparently a more democratically sound alternative called Nash learning from human feedback. Dan’s also pointed to a growing field called differentiable social choice — treating voting rules as learnable models — and reckons it’s the neighbourhood alignment research should be moving into.

4 The bits and pieces

4.1 Ageing

The ageing notes got a proper overhaul, and the headline is not what the supplement crowd wants to hear: exercise beats everything else by a margin that makes the rest look like rounding error. There’s also a methodological problem Dan’s straightforward about — almost nobody in the literature can actually tell whether an intervention is slowing how fast you’re ageing or just giving you a one-off health boost, and the difference matters enormously for the maths. He’s settled on a concrete decision rule ($100 per expected day of healthy life; creatine retained on grounds of cheapness) and devoted a proper section to the vitalism movement — biohackers, Blueprint, Don’t Die — pegged fairly accurately as 21st-century vital-force mysticism with NAD+ precursors where élan vital used to be. The supercentenarian data is also a worry: most record-setting longevity claims cluster with missing birth certificates and pension fraud, not any actual biological signal.

4.2 Attention Deficit (Hyperactivity) Disorder

The backdoor transhumanism observation is the one that stuck with me: the drugs used to treat ADHD are exactly the drugs people try as general cognitive enhancers, which means anyone who rejects ADHD medication on anti-medicalisation grounds but would accept a productivity nootropic has quietly done transhumanism through the back door without noticing. Dan’s also clear-eyed about the baseline — the comparison isn’t “medicated ADHD versus neurotypical” but “medicated versus unmedicated ADHD brain actually navigating traffic, relationships, and tax returns” — and the actuarial gap between those two is supposed to be something like thirteen years of life expectancy, although Dan is not buying it is that big. The section on tolerance is worth reading separately: the euphoria wears off, but the therapeutic effect mostly doesn’t, so the drugs aren’t that fun but they are useful, which I guess is why society lets people take them.

4.3 Editing images with machine learning

New post, and handy. The ML image-editing landscape has settled enough to map: a handful of instruction-following editors you can give plain-English instructions to (FLUX.1 Kontext for open-weights, Gemini and GPT-Image-1 for hosted, Adobe Generative Fill for anyone already living in Photoshop), plus specialist tools still worth knowing for specific jobs — background removal, upscaling, face restoration. Dan’s separated this out from the generation notebook because the distinction between “edit what already exists” and “conjure from a prompt” has become meaningful enough to warrant its own page.

4.4 Editing images using code

The programmatic editing notes got extended with a GIMP scripting reference — turns out GIMP can run headless from the command line, which is genuinely handy when you need to batch-process images without clicking through a GUI every time. Also added: OpenCV for classical computer-vision operations, and macOS’s built-in sips command for a quick format conversion without installing anything. Good reference page, more complete now.

4.5 Czechia

Dan wrote up his mental map of Czech history before visiting Prague — everyone and everything he’d ever heard of, to see what he’d miss and which prejudices he’d reveal. Good yarn: the Hussites pulling off a proto-Protestant Reformation a century before Luther with better armies and early handguns; defenestration recurring as a constitutional mechanism across five separate centuries; Karel Čapek coining the word “robot” in a 1920 play; Mendel’s pea genetics established in a monastery vegetable garden, then debated for reliability ever since. There’s a language survival kit at the end, and the observation I didn’t know: Václav and Wenceslaus are the same name — the Latin form got preserved for kings while living Czechs kept the Slavic one.

4.6 Zürich

Short placeholder for a city Dan lived in for a couple of years — an AI safety coworking space and GPS coordinates for outdoor workout stations so far. He says there’s more to say. I believe him.

4.7 London

The new transport section is about bike infrastructure in central London, which Dan reckons turns out to be world-class — he wasn’t expecting that. The regional trains, though: at King’s Cross the platform isn’t announced until six minutes before departure, which rules out buying a sandwich and creates a mad sprint for seats; his theory that the trains are powered by quantum indeterminacy and need their wave-function collapsed before they can commit to a destination doesn’t explain why there’s no running water in the washrooms, but it’s the working hypothesis.