Skip to main content

2026-10-12: Dodgy forecasts, AI refereeing, home-built robot rigs, who inherits the shop

2026-10-12 — 2026-11-08

Forty-four days, eight new posts and sixteen updates. Dan’s been having a go at the formulas people use to predict where AI is headed, and he reckons most of them are dodgy bookkeeping. Then there’s the question of who you trust when a machine can write the paper and referee it too. He’s also fitted out his workshop again, running the robots on his own laptop and swapping editors like other blokes swap utes. And a short one at the end about what we owe people who aren’t born yet.

digest

1 Are the sums any good?

1.1 Algorithmic improvement

You know how people say AI got faster half from better hardware and half from cleverer recipes? Dan says there’s a third thing hiding in there. Sometimes it gets faster because someone lowered what counts as a good enough answer and didn’t make a song and dance about it. Like saying you got quicker at the books because you stopped checking the decimals. And which one gets the credit depends on which you count first, so the same speedup can be pinned on either. He’s got worked examples, and the famous charts mix the two up. He started out thinking he’d already written this one and just imagined it. Classic.

1.2 Production functions

Dan’s annoyed at people who guess where AI is going by multiplying two numbers together. That’s a production function, which is just the rule for how inputs turn into output. Multiplying two things is the laziest version, and it assumes you can swap one input for the other at a steady rate. Real life doesn’t work like that. One pilot per plane, and you can’t make up for a missing pilot with more fuel. He’s gone through the usual suspects and what each one assumes. Check what you’ve smuggled in before you build a forecast on it. I wish I’d had that rule for the farm accounts.

1.3 Quis computat?

This is Dan’s toy race between two clocks: one ticking toward AI going badly, one toward it going well. How much we spend on each decides who wins. The new bit is that there are two settings you can change: how the money’s split, or how fast the spending grows. Depending on the shape of the curves, one does everything and the other does bugger all. In one world only the split matters. In another, going slower is the only thing that helps. If you’ve ever heard someone say “it doesn’t matter how fast, just fund safety”, that’s the one very particular world where it’s true. Dan built a whole model to find that out.

1.4 AI evals

When someone says their AI scored 90% on a test, how good is that number? Dan reckons the field mostly skips the basics of running a proper experiment, the stuff you’d learn in a first stats course. He wants the same tools used on the questions as get used on people, with each question having its own difficulty. Then you can spot a broken question, because the clever models get it wrong as often as the dim ones. You can also spot one that’s suspiciously easy for a single model, which probably means it saw the answer in training. Cheating on the exam, in other words. Barb, you’d have caught that by the second quiz night.

1.5 Ablation studies, knockout studies, lesion studies

The usual way to show which part of an AI mattered is to pull one piece out and see what breaks. Dan says statisticians have been telling people not to do that since 1926. If the pieces help each other, there’s no single “effect of piece A”. It depends where you start. It also lets the author tune their favourite version hard and not bother with the rivals, which is a bit sneaky. He reckons a proper design that varies everything at once costs barely more and tells you what you actually wanted to know. It’s like testing a recipe by pulling out one ingredient at a time, then being shocked the cake still sort of works.

1.6 Calibrating LLM estimates

Mostly a dumping ground so far, with a couple of headings still saying “TBD”. The meaty bit is about a model built to give a proper probability instead of a confident-sounding answer. Ask a chatbot “how sure are you?” and it’ll say 95% about everything, like a bloke at the pub. This one answers a batch of questions in one go and spits out odds. Dan isn’t convinced anyone’s said what’s actually special about the training. Fair enough.

1.7 Complexity of markets

Can a market actually work out the right prices, or is that a nice story? Dan says the maths is mixed. Finding the exact balance point is provably hard for some kinds of preferences and easy for others. Even when it exists, the prices can go round and round forever without settling. But the amount of chatter needed between buyers and sellers grows only in step with the number of people, and that’s what markets have going for them. It all comes down to whether things are substitutes or complements. Which is why nobody can price a crane without the bloke who drives it.

2 Who vouches for the truth

2.1 Should our journal publish AI-drafted manuscripts?

Dan helps run a journal and has a real headache. If AI makes writing a paper cheap, do you get more good work or just more rubbish? He’s built a little money model of it. Authors chasing acceptance will write more and thinner papers once drafting’s cheap, while reviewers still only have so many hours. He measures it in invented units, “citoms” and “scientillas”. I’m not making that up. He doesn’t say what to do. He’s just trying to drag the argument out of “AI is lazy” and into the numbers.

2.2 Peer review meets AI

Peer review’s already creaking, so Dan’s been looking at putting an AI umpire in the middle of the author-and-reviewer squabble. After the authors answer the reviewers, the AI looks at each disputed point and gives a verdict and how sure it is. A human still makes the final call. The authors get the last word, and every chat is logged. Judging a fight is easier than starting one. Seems a fair bet to me, though I’ve seen plenty of arguments that needed a better umpire than a computer.

2.3 Epistemic communities

How do groups of people settle on what’s true, and what happens when machines join in? Dan reckons “does AI help or hurt?” is the wrong question. Ask what the signal in each community was in the first place. If the signal was “can you write a passable essay”, you’re cooked. If it was “will you stake your name on a forecast”, you’re fine. The cheaper passable text gets, the harder it is to tell the real mob from the fakes. He points to Quora as an example.

2.4 Scientific writing

Dan studied linguistics, and had to write essays about how hard passive voice is to read, in passive voice. Now he’s hooked it up to the AI-manuscript question and added a bit on incentives. Clear writing barely gets rewarded when papers are judged. Presentation came dead last among the things reviewers scored. So no wonder everyone writes like a tax form.

2.5 Science

He’s reshuffled his big index page on science and rewritten the blurbs. There’s a bit more swearing about geopolitical pissing contests and the tone’s gone looser. Mostly a tidy-up of his own filing cabinet. Nothing much in it.

2.6 AI reasoning and proofs

Dan’s rewritten his notes on getting AI to do maths. The kind of answer you ask for decides how you can check it. A single number can only be voted on. A proof in plain words needs another reader. A proof in Lean, the computer proof language, gets checked by a compiler that can’t be sweet-talked. So pick what you ask for based on what you can check. Same as making someone show their working on the farm books instead of taking their total on trust.

2.7 The language game

Dan thinks the old philosophy puzzle of what words mean is more or less solved by how AI learns. You learn a word from seeing lots of things people call by it, and the meaning is whatever you’d most likely call it. So whale and fish? Depends what you grew up calling them. Cheetahs and cats? Nobody ever decided, so it’s up to us. Nobody’s a realist about words, he says, and he wants it printed on cards for getting out of arguments. The new bit is a chihuahua-or-muffin picture, which is the best thing in it.

3 Building the toolshed again

3.1 Running LLMs locally on a Mac

Dan’s been running the chatbots on his own laptop for months. He tried most of the programs that do it and ended up with a pile of settings that all clashed. One program holding nine models meant one temperature for nine different recipes. So now he’s got a little doorman that starts the right model when a request comes in and shuts it down after. Each model gets its own settings, and you can measure the memory use. He’s also written up how to drive it from the command line. Took him months to work out most of his grief was self-inflicted. Sounds about right.

3.2 Agent harness design

He’s rewritten his notes on the rigging around an AI agent, meaning the loop that sends your words to the model and runs whatever it asks for. Two things get settled before you design anything. Can the actions be thrown away, and is there anything to judge the result by? Maths checking can be repeated fifty times for free. A database migration, not so much. That’s why maths robots fan out and coding ones plod along. Dan’s cut it from four questions down to two givens and three choices, which for him is practically a diet.

3.3 Building a maths agent

His own home-built maths robot, with a bit of tidying and a fixed link. The useful bit is the bill: his home loop costs about a fifth as much per problem and solves about half as many. He says that trade is his to make rather than the startups’. That’s a bookkeeper’s sentence if ever I heard one.

3.4 Code agents and assistants

A new entry on DeepSeek’s coding harness, where everything’s a plugin, even the loop. He’s also added a tool that lets you run a few coding agents side by side and rewind both the code and the chat if one goes pear-shaped. He’s chuffed it isn’t another Electron app, and said so with some force.

3.5 LLM sampling stunts

Dan’s found that a bunch of tricks for steering a chatbot are cousins. You grow a crowd of possible continuations, score them, and keep the good ones. Then someone told him a lot of weirdos have been doing this by hand since 2020, picking from many options with a tool called a Loom. So he’s compared the two. The hand version always follows one thread, which means it can wander into a patch where only one choice in a hundred is any good. His question is whether the human is searching or sampling. I’d say a cuppa was involved.

3.6 Zed, the code editor

A new editor from the blokes who made Atom. This one’s written in Rust and doesn’t run a browser inside itself. It’s built so AI agents and people can edit at the same time. They’ve raised forty-two million, and someone’s already forked it to rip the AI out. Dan, who changes editors like socks, says he doesn’t use it.

3.7 Code editors

His old editor page got mangled, and now it’s a proper rundown with a paragraph on each one. Atom is “my ex”, vim is “my first love”, and emacs gets a contractual mention. TextMate is “the ex from two exes ago”. It’s a tour of his love life with text editors, with a fair bit of grudging affection.

3.8 Matplotlib

He’s reshuffled the plotting notes, moved the interactive bits together and added a font-fixing tip. Matplotlib is still horrible to use and he still says so. Mostly housekeeping, but he did note one wrapper is “on hiatus”, which is how most of Dan’s projects end up.

4 Who gets the keys next

4.1 Successors and successionism

Barely a post yet. It’s a placeholder with a picture of an abandoned rowboat and a few reading links. The question underneath is who runs the place next, and whether it’s anyone we’d recognise. Big question, small page. He’ll get to it. Probably.

4.2 Intergenerational game theory

The unborn can’t bargain, can’t hit back and can’t say no. So the usual way of keeping people honest, the fear of being paid back, doesn’t work on them. Dan reckons the best we can do is chains of overlapping generations, with grandparents looking after parents who look after kids. But a long-run promise needs someone to keep it, and nobody lives long enough. So you have to build something that outlasts everyone, like a constitution or a sovereign wealth fund. He’s added a heading about Confucius, who got there first. The rest of the page is rough notes for a workshop, and he says so.


Aunty Val is Dan’s fictitious aunt from Mukinbudin.