2026-01-20: Imprecise Bayesianism, Generative AI workflows, Causal inference, AI evals, travel notes
2026-01-20 — 2026-01-20
Over the past nine days the lad’s been flat out: five new posts and eleven updates. He’s been wrestling with imprecise Bayesianism — that means admitting you don’t always know a single tidy probability and working with ranges instead — and put together a hands-on ‘Generative AI workflows and hacks 2026’ for getting models to behave. There’s a new piece on causal inference in learning to act — that is, figuring out what actually causes outcomes so decisions aren’t just guesses — plus updates across causality, reinforcement learning, probabilistic-graph visuals, AI evals and even his scientific writing. Oh, and he slipped in travel notes from London and the San Francisco Bay Area. Plenty here if you like knowing why models do what they do, not just watching them spit out answers.
1 Newly published
1.1 Imprecise Bayesianism
Fair dinkum: Imprecise Bayesianism is just Bayes without pretending you can pin one exact probability — you let your belief be a range or set of distributions when your model’s likely wrong. Dan walks through the old-school set-prior idea (update each prior then use a maximin or worst-case decision), PAC-Bayes bounds that give probability-backed generalisation guarantees for ensembles, and the newer-sounding infrabayesian bit with ‘infradistributions’. It matters because real models are messy, and ranges or worst-case rules stop you being overconfident when stuff goes pear-shaped. He’s frank about what he’s actually used and points to tools and references if you want to poke under the hood.
1.2 Generative AI workflows and hacks 2026
Righto, Dan’s put together a grab-bag of hacks to stop AI outputs turning into a messy tangle when you copy them. He points out that OpenAI now spits out ugly HTML for maths and that Google Deep Research loses inline links, then lays out practical fixes: use a proper client that keeps markdown (like Jan), a tiny macOS clipboard script to convert HTML→Markdown, or his Deep Research client that preserves links and maths. It’s the sort of tidy little lifesaver that saves you an hour of cleanup and a fair few swear words when you actually want usable notes with code and equations.
1.3 London
Fancy a gossip about trolleys and class? Dan walks London by its shops, showing how supermarket choice — Waitrose, M&S, Sainsbury’s, Tesco down to Aldi/Lidl — and department stores like Harrods or Selfridges act as blunt little signals of where someone sits on the social map. He also flags Kew Gardens as the old empire’s botanical HQ, just to remind you this city wears history on its sleeve. It’s useful because it turns everyday shopping into a readable clue about taste, postcode and money worries — and why even wealthy-looking households might have an Aldi bag now the bills bite.
1.4 San Francisco Bay Area
You’d swear San Francisco was half dream, half building site. Dan wanders the Bay Area, ticking off why it still pulls visionaries and weirdos, name‑checking quirky institutions and lesser-known people worth watching, then has a proper moan about the state of the basics. He digs into sewers, stormwater and ageing PG&E gear — the sort of dry, grubby stuff that actually breaks things. Why care? Because this is where tech gets tried, and if the pipes and power are dodgy the fancy ideas mean squat.
1.5 Causal inference in learning to act
Grab your cuppa — Dan’s been poking at how machines learn to act, but with a causal twist. In plain talk: learning to act is really about ‘what if’ questions — what happens if I do this — so it’s natural to treat it like cause and effect, not just pattern-spotting. He walks through how reinforcement learning fits that idea, points to bandit problems as the simple starting point, and warns that learning from other folks’ logged data can be dodgy because hidden confounders sneak in — which is where offline learning can actually help. He also points at history-based RL and throws in a note about predictive coding and earlier work. Why bother? Because if you want machines to make sensible, safer choices from messy real-world data, thinking in causal terms stops them being fooled by mere correlations — and that’s the bit Dan wants you to notice.
2 Updated
2.1 Scientific writing
Stone the crows — Dan’s gone and added a proper ‘Actual advice’ section with concrete, usable tips and a big pile of links you can actually use. He’s also dropped in ‘Machine learning’ and ‘Mathematical’ sections that tell you how to write ML stuff and tidy up maths so readers don’t switch off. ‘Miscellaneous invective’ is his charming rant about the worst bits of academic gobbledegook. Overall, it moves the notes from moaning about bad style to giving people real, practical help for writing technical things clearly.
2.2 Reinforcement learning
Well, would you look at that — Dan’s added three proper chunks: History-based RL, Reinforcement learning and causality, and a Variants roundup. The Variants bit gathers the oddballs: reward-free ideas like empowerment, generative-action setups, hierarchical RL and even diffusion-flavoured decision-making, with pointers to the other notes. History-based RL gives space to agents that use memory (think MENACE and more modern history-driven tricks). There’s still the hands-on bits too — a plain PyTorch REINFORCE example if you want something to run and poke at.
2.3 Utility and evolutionary fitness
Righto — Dan’s added a proper prelude treating natural selection as a kind of statistical inference, plus a new section tying the replicator equations to that inference way of thinking. He’s made the math clearer that Malthusian (log) fitness works ‘as‑if’ like a utility — same idea as the Kelly criterion — so evolution ends up risk‑averse (bet‑hedging) because growth is multiplicative. There’s a blunt new chunk saying fitness landscapes aren’t real, and he explains why frequency dependence, game‑theory cycles and multi‑level selection mess up the simple ‘climb a peak’ story.
2.4 Bikes
Oh good — Dan’s added a Brompton section. He says the Brompton is still the folding bike to beat: folds small enough to skirt that 158 cm baggage limit, rides brilliantly, and has a healthy secondhand market. He warns they’re about 50% dearer in Australia so don’t buy here, and drops pro-tips — IKEA DIMPA bags fit, Carradice makes nice aftermarket luggage, and take care with hard cases (they can push you into oversize fees).
2.5 PIBBS x ILLIAD Research residency January 2026
Righto — Dan’s stuck a proper Lectures section in and added a few meaty bits: ‘Implicit themes’, ‘Computational no‑coincidence conjectures’, ‘Reinforcement learning meets SLT’ and a note on “Model Local Learning”. It’s still the write‑up of the London residency (5 Jan–13 Feb 2026), but now has short notes on who said what. Standout: Chris Elliott’s slot shows how Singular Learning Theory can be brought into reinforcement learning, with a Gibbs‑posterior style policy idea — he’s honest that it rests on some strong assumptions. George Robinson’s talk on computational coincidences is in too, and the implicit‑themes section pulls together recurring threads like partial info breakdown, edge‑of‑chaos and embedded agency.
2.6 Causal inference in highly parameterized ML
Crikey — Dan’s added a proper “Causality with agency” section and a fresh chunk on “Implicit causality in foundation models,” and he’s stuck pgmpy and cause2e on the page so coders have something to grab. He’s calling out practical tools like DoWhy and TETRAD for building and testing causal graphs in real code. Benchmarks such as CauseMe get a shout for handling dataset shift, so it’s not just theory. The through-line is trying to make causal graphs actually useful around big nonparametric models — neural nets and foundation models — rather than hand-waving about them.
2.7 Causality, agency, decisions
Righto — Dan’s added some proper meat on turning causal graphs into things that can hold decisions. The new ‘Basic Mechanization’ bit explains influence diagrams where decisions are treated probabilistically, so agents can be modelled as choosing rather than being fixed bits. There’s a ‘Mechanized Multi‑agent DAGs’ section showing how several deciding agents can be wired up and affect each other. And a new ‘Tooling’ note points to pycid, a Python library you can actually use to compute with these causal influence diagrams — handy for thermostat feedback examples and multi‑agent tinkering.
2.8 Melbourne / Naarm
Righto — Dan’s added a proper Urban planning section and reshuffled the Naarm stuff into a new ‘…as Naarm’ chunk that actually talks pronunciation and why name-arguments are mostly a time-waster. He’s swapped the old electronic-music heading for a clearer ‘Where’s the electronic music?’ with nuttier local notes. There’s a firmer line about Buruli — the flesh-eating bacteria — creeping in, and a shout to join YIMBY Melbourne if you like poking at housing policy. Little practical touches too: Spa Wars gets a clearer mention, and the audio-repair notes now read like someone who’s used the services. Overall it feels less twee and more useful for anyone who actually lives here.
2.9 Visualising probabilistic graphical models
Righto — he’s added a pgmpy section. It flags that pgmpy can render model diagrams and hooks into pygraphviz, networkx or daft for plotting. Since the page is about drawing probabilistic graphical models with plate notation, inline maths and exporting to SVG/PDF, pgmpy gives a tidy Python route for clean vector output.
2.10 AI evals
Stone the crows — Dan’s added a proper “Statistical methods for evaluating AI systems” chunk calling out how many evals have lousy stats (no RCTs, weak power, multiple-comparisons mess) and points to BetterBench and repo tools. The rest tightens the benchmarks vs evals split and flags EvalEval, Inspect, plus using human judgements and user logs for real-world checks.
2.11 How to reduce government spying on you
Blimey — Dan’s added a ‘Bluetooth is cursed’ section telling you to turn Bluetooth off unless you really need it, with links showing headphone‑jacking and dodgy chip‑vendor threads. It’s tacked onto his how‑to for dodging state spying, alongside tips like SecureDrop, USB condoms and hardened OSes.