2026-06-26: Training models on your own prose, predictions that push back, distributed knowing, the local AI workbench

1.1 Fine-tuning danbot

No matter how many prompt tricks Dan tried — banned-words lists, in-context examples, multiple “rewrite this in my voice” passes — Claude kept coming back smooth, signposted, breathlessly enthusiastic in the flat bits. He’s hit the ceiling on what prompting can fix. So he’s training a small “LoRA adapter” on actual before-and-after edits, baking the style into the weights instead of pleading for it each time. He’s already shipped a logger that captures every AI-draft → Dan-rewrite as he works, so the training corpus builds while he’s doing other things. Knowing Dan, he’ll have three variations of this running and none of them finished by the time the next update lands.

1.2 Fine tuning foundation models

The theory companion to the danbot project, with a new section on abliteration. Researchers found that refusal behaviour in chat models isn’t scattered through the whole network — it runs along a single direction in the residual stream. Run the model over harmful and harmless prompts, take the difference of the means at each layer, and that’s your refusal direction. Wipe it out — project it from the weights or subtract it from activations at inference — and the refusals go with it, no retraining required, nothing beyond a few hundred contrastive prompts. As safety mechanisms go, a single erasable vector is a thin coat of paint.

1.3 Community sovereign AI compute

The case for owning your own box has got sharper. Today’s generous AI pricing is step one of enshittification: you get locked in, switching costs rise, terms narrow. There’s the Anthropic-Pentagon incident — the Trump administration branded Anthropic a “supply chain risk to national security” when its CEO refused unrestricted military access — and the fact that AI datacentres got bombed in the Iran war, and Australia’s eighteen submarine cables keep getting sabotaged. The line to a box in your city doesn’t cross any of that. This post makes the case; the technical companion below tells you what to put in the box.

1.4 Sovereign compute for small collectives: technical implementation guide

The practical companion to the one above, with a good model roster update. The new addition is Xiaomi’s MiMo-V2.5: fits inside the reference hardware’s memory budget with room spare for the cache, runs faster per token than the Qwen baseline, and handles text, images, video, and audio in one model under a plain MIT licence. Same de-censoring procedure applies as to any other Chinese open-weight model — the abliteration technique from the fine-tuning notes above works the same regardless of whose model is under the knife. Abliteration tooling sometimes lags fresh architectures by weeks, so check before assuming it’s a simple download.

2.1 Performative prediction

A credit model that rates you high-risk might cause your default, by giving you an interest rate you can’t service. The model isn’t watching the world passively — it’s pushing it. Deploying a model changes the distribution you’d see if you retrained, so the usual training gradient doesn’t work cleanly. Two ideas fall out of that. A performatively optimal model accounts for its own downstream effect. A performatively stable model is already calibrated for the world it creates — retraining won’t budge it. Those two aren’t the same model, and naive retraining converges to the stable one, which can be perfectly self-confirming without being good. He pulls in Scott’s Seeing Like a State at the end: a classifier is a form of state legibility, and populations don’t sit still under classifiers. Which the ATO has absolutely figured out.

2.2 Algorithmic collective action

Flip side of the performative prediction post, more or less. If a platform retrains on whatever data it sees and you coordinate with even a small group to shape that data, you can steer what the model learns — the effect compounds across retraining rounds. The second half goes into coalition formation between algorithms, using the nucleolus from cooperative game theory: allocate payoffs so the most-aggrieved sub-group is as un-aggrieved as possible, and agents trained under this objective autonomously split into stable sub-teams instead of being jammed into a grand coalition where someone’s always getting a raw deal. The multi-agent RL post below has the implementation details.

2.3 Informational empowerment

The formal definition of an agent’s drive to keep its options open: the mutual information between its actions and its future states — roughly, how many distinguishable futures it can reliably reach from here. As an RL objective it’s less useful than it sounds because estimating it is hard, there are no decent optimality results, and it tends to create more problems than it solves. It keeps coming up when people try to formalise what “power-seeking” actually means though.

2.4 Power-seeking

The other flavour of empowerment. Instrumental convergence is the argument that capable agents will resource-acquire and self-preserve regardless of their specific goals, because control over the environment is useful for almost anything. You could just read it straight off the empowerment maths — an agent maximising its influence over future states is a power-seeker — but Dan reckons the identification isn’t clean. In open-ended or multi-agent settings it’s hard to even say what “future influence” means, let alone compute it neatly. The worry is real; whether the empowerment maths captures it is a different question.

3.1 Cooperation through uncertainty

Animals can’t run PCR tests to check who they’re related to, so kin recognition is a signal detection problem with noisy cues and costs and benefits measured in expected gene copies. What you get is an acceptance threshold — an optimal cutoff on the cue where helping becomes worthwhile in expectation, and that threshold leaves calibrated errors in because fixing them costs more than they’re worth. When kin are common and snubbing one is expensive, go permissive. When kin are rare and altruism costs a lot, go choosy. Same logic turns up everywhere — brood parasite defences, host-pathogen sorting, anywhere an animal sorts a stream of individuals under uncertainty. Which, come to think of it, describes Thursday bridge night fairly well.

3.2 Wisdom from the madness of crowds

Dan’s been reading some researchers who treat beliefs as mechanisms that do work — they take in observations, make predictions, drive decisions — and connects this to the pragmatist tradition: beliefs have “cash value,” meaning what holding them actually gets you. Point is, anyone who can’t re-derive everything from scratch — which is everyone — can rationally import other people’s beliefs instead, and there’s a principled way to work out when copying is cheaper than reasoning.

3.3 Homunculi all the way down

The main new addition is Hofstadter’s strange loops. The argument from I Am a Strange Loop: the “I” isn’t the thing that has a self-model, it is the self-model. Get complex enough and a system can’t avoid building a symbol for itself — same reason Gödel could make Principia Mathematica talk about Principia Mathematica. The loop closes: the coarse self-symbol steers the machinery that implements it, which then re-represents what it just produced, around and around. The coarseness isn’t a bug — a perfectly accurate self-model would let you always compute the predictor-defeating action, which breaks things; a reduced-rank one converges, the way video feedback settles into stable spirals because the camera is lossy. Dan’s got no theorem for any of it, which at least makes two of us.

3.4 Neural nets that do symbolic mathematics, logic and other reasoning tasks

Much expanded. There are solvers and provers, and they’re different things. A solver answers maths problems in natural language, possibly running Python to check its working. A prover emits Lean 4 formal proofs, handing them to a compiler that either accepts or rejects them — unlike a majority vote, a compiler can’t be fooled by a self-consistent wrong answer. The test-time scaling section is properly written up too: sample many candidates at non-zero temperature, combine them, and you can beat a much larger single-shot model. The applied post below has the actual workflow; this one is the theory for why the two families behave differently.

3.5 AI tutoring

Dan’s been trying to turn Claude into a Socratic tutor rather than an answer machine. The problem: helpful and pedagogical are different objectives. Claude is trained to give you what you asked for; a tutor is supposed to make you work for it. He’s got a long system prompt that constrains the model into a reluctant stance — questions first, climb a reluctance ladder before giving direct answers. His review after a few weeks: it works but it slips. Claude keeps reverting to summary mode whenever it gets a weak signal like “got it” or a long pause. Still better than plain chat for learning maths, he reckons — which as endorsements go is not exactly ringing.

4.1 Running LLMs locally on a Mac

Full guide to the local LLM stack on Macs — runtimes, quantisation formats, desktop apps, headless servers, memory management. Three compute backends to know: PyTorch+MPS (widest coverage, new architectures arrive here first), llama.cpp (fast, broad, the engine under Ollama), and MLX (fastest on Apple Silicon but lags new models by months). Osaurus gets the nod as the sensible default — Swift-native, no Electron, full agent harness, MCP in both directions. One gotcha: macOS caps how much RAM Metal can wire to around 75% of your total by default, so a 128GB Mac effectively tops out at 96GB for model use without a workaround. Push it too far and the machine kernel panics. I don’t know exactly what that means but it sounds expensive.

4.2 AI agents, applied

New vocabulary guide to the agent stack: model, runtime, server, harness, frontend — what each layer does and why it matters which one you’re looking at when something breaks. Paired explicitly with the Mac post above as the practical backend guide. Read the security section. Agents that write and run code and reach the network have what one linked source calls “the lethal trifecta” in play, and permission-prompt gates are security theatre once you’re clicking through two hundred of them without reading. A good sandbox and audit logs beat permission prompts as a safety strategy. Dan’s right about that, and will still click through all two hundred.

4.3 Maths and proof models, applied

The workshop companion to the AI reasoning theory post above. Dan’s built a ladder from “just ask your agent to do maths” up through building your own generate-run-verify loop with fan-out sampling — That’s voting on the answer to you and me.. Fan-out is what you adjust at each level: sample many candidates in parallel, combine them, and you beat a larger single-shot model for the same compute. For formal proofs, the loop swaps the Python executor for a Lean compiler — exactly right or exactly wrong, no clever averaging. Working repo is up at his GitHub. He’s built more infrastructure than strictly necessary, as you’d expect, but this time it’s documented.

4.4 Code agents and assistants

Major restructure. Now cleanly split between cloud-tied commercial harnesses — Claude Code, Cursor, GitHub Copilot, all vendor-dependent — and model-agnostic offline ones you can point at a local backend. GitHub Copilot gets a backhanded compliment: the original Codex engine was genuinely excellent, and the general-purpose replacements still haven’t matched it years later. The offline picks are OpenCode and pi, a minimalist TypeScript harness whose whole system prompt fits under a thousand tokens. Read it alongside the Mac post since the local backend is what the offline harnesses actually run against.

4.5 PDF ingestion

PDF converters that use a single AI vision model to transcribe a whole page silently corrupt maths. Dan ran the field over equation-dense papers and found the Docling vision pipeline just turned log-fraction terms into different wrong fractions — valid typesetting, wrong maths, exactly the kind of error you’d never catch without checking. Specialist pipelines that route equations through dedicated recognisers (marker, MinerU) came through clean. If you’re feeding AI your technical papers and asking it to say anything about the results, check what it thinks the equations actually say.

4.6 Generative AI workflows and hacks 2026

Updated with a comparison table of which AI providers train on your prompts — most paid APIs don’t by default, consumer free tiers usually do, and “no training” and “zero data retention” are two different promises the industry has helpfully started conflating. The “going agentic” section is still in progress. Of course it is.

4.7 Front-end clients for AI image models

He’s added Chinese community sites to the platform comparison — ModelScope, LiblibAI, Shakker — since so much of the open-weights frontier right now is Chinese-made (Qwen-Image, Z-Image Turbo, Wan video), and the LoRA catalogues for those models are richest there, sometimes weeks ahead of CivitAI. Draw Things gets a proper walkthrough: it’s the only local client here that both loads and trains LoRAs on-device. New model comparison entries too, so FLUX.1 dev, schnell, Z-Image Turbo, and Qwen-Image each get a proper description rather than a footnote.

4.8 Backups

Dan’s finally had the portable hard drive conversation that everyone who’s used them eventually has. Five drives, four laptops, two years, same result: spontaneous disconnects, data corruption, drives that appear to mount but won’t read or format. His diagnosis: a 5TB bus-powered spinning drive sits at the limit of what USB can power at spin-up, and the Micro-B connector was chosen for cheapness over robustness. The fix is SSDs, but 4TB portable SSDs currently cost about a thousand dollars because AI datacentres are hoovering up the NAND supply. So he’s stuck with the cursed spinning drives for now, which puts him in good company.

4.9 Generative art with language+diffusion models

The where-to-find-models section now covers Chinese platforms since so much of the current frontier is Chinese-made and the LoRA catalogues for those models are richest there. He’s added a model comparison: FLUX.1 dev for photorealism and text legibility, schnell for faster iteration, Z-Image Turbo for quick generation with good typography, Qwen-Image for infographics and labels. He’s also written up the difference between a LoRA (small adapter, bolts a style onto a base model without retraining it, the sensible choice most of the time) and a full fine-tune (whole checkpoint, several gigabytes, for when you really mean it). Overlaps in places with the image clients post; between them you get both the what and the where.

4.10 Awesome illustrators and printmakers

Nothing to do with AI. Dan went down a public domain engraving rabbit hole and came out with a personal canon: Dürer, who drew the most famous rhinoceros in Europe without ever seeing one — armour-plated, riveted, with a spare horn it never had. Hogarth, who kept a pug named Trump and painted the two of them together as a self-portrait. Gillray, whose Bonaparte caricatures may have permanently convinced history the man was short. Piranesi’s prisons that climb forever and never arrive. It is a list of pictures he likes, organised with more care than many of his technical posts. I’ll allow it.

Aunty Val is Dan’s fictitious aunt from Mukinbudin.