PIBBS x ILIAD Research residency January-February 2026

2026-01-02 — 2026-02-27

Wherein a Shoreditch residency at the London Initiative for Safe AI is recounted, held 5 January–13 February 2026, with lectures ranging from information decomposition to singular learning theory.

AI safety
machine learning
neural nets

I was in the first cohort of the Principles of Intelligence Research Residency. It ran from 5 January to 13 February 2026 in London.

The pilot of the residency was successful. There will be some follow on programs that you yourself might wish to apply to.

Figure 1

The residency was based at the London Initiative for Safe AI in Shoreditch. It was a lot of productive chaos.

Here’s the blurb:

tl;dr: PIBBSS and Iliad are announcing a Research Residency in applied mathematics and AI alignment. Competitive candidates are PhD or postdoctoral (or equally experienced) researchers in math, physics, or a related field. Our target outcome is for Researchers-in-Residence to continue research after the Residency by securing philanthropic funding to either start new research groups or to support other research projects.

I can now announce the major outcome of this project for me which was the founding of the Alignment Journal; for more details see the announcement post.

1 Lecturers

Many! This long list is just a partial sample of the many luminaries who contributed. I missed a few lectures because there was just too much going on; Many of the connections I made while I was there led to projects, and there was not time to do everything.

2 Themes

No pretence of completeness—just the ones that seemed relevant.

2.1 Information decomposition

Several attendees (e.g. Jansma (2025); Kolchinsky (2022)) are interested in a principled use of partial information decomposition (Williams and Beer 2010).

2.2 Edge of stability, edge of chaos

Another theme was the edge of stability and the edge of chaos.

2.3 Embeddedness

See the embedded agency post.

2.4 “Model Local Learning”

Lucius Bushnaq of Goodfire takes a crack at building intuitive models for “generic learners”.

2.5 Computational no-coincidence conjectures

George Robinson presented some work on computational coincidences.

2.6 Reinforcement learning meets SLT

Chris Elliott presented Elliott et al. (2026), which applies Singular Learning Theory to reinforcement learning. I liked the construction where the policy acquires a global Gibbs posterior; as Chris noted, it makes some strong assumptions. Still, I learned a lot.

2.7 Intentionality and agency

So much on this topic (Biehl and Virgo 2023; Hafner et al. 2022; Meulemans et al. 2025).

I synthesized some into intentional language is OK, causally embedded agency, and agency embedded in causality.

I’m still trying to process Martin’s presentation, based on Biehl and Virgo (2023) and Virgo, Biehl, and McGregor (2021). Martin pulled in lots of fun tools (string diagrams, category-theoretic systems, “lumping”, as in (Teza, Stella, and GrandPre 2025) of coupled Markov systems…) to argue about internal model theorems and the intentional stance.

3 Singular learning theory

Jesse Hoogland gave an introductory lecture, but I missed it.

Edmund presented some results from his thesis, connecting minimum description length and singular learning theory via a round-trip through Jeffreys’ priors (B. S. Clarke and Barron 1990; Bertrand S. Clarke and Barron 1994) and the Kraft–McMillan inequality of coding theory.

Max Hennick presented loads of provocative parallels between the phase transitions of statistical learning and physics, like the phase-field theory of crystal growth.

Also—an aside from Lucius: From SLT to AIT: NN generalization out-of-distribution — LessWrong.

Fernando Rosas presented his work on internal models as transducer reduction.

4 References

Biehl, and Virgo. 2023. “Interpreting Systems as Solving POMDPs: A Step Towards a Formal Understanding of Agency.” In.
Clarke, B.S., and Barron. 1990. “Information-Theoretic Asymptotics of Bayes Methods.” IEEE Transactions on Information Theory.
Clarke, Bertrand S., and Barron. 1994. “Jeffreys’ Prior Is Asymptotically Least Favorable Under Entropy Risk.” Journal of Statistical Planning and Inference.
Demirtas, Halverson, Maiti, et al. 2023. “Neural Network Field Theories: Non-Gaussianity, Actions, and Locality.”
Ebtekar, and Hutter. 2023. “Foundations of Algorithmic Thermodynamics.”
Elliott, Urdshals, Quarel, et al. 2026. “Stagewise Reinforcement Learning and the Geometry of the Regret Landscape.”
Garrabrant, Benson-Tilsen, Critch, et al. 2020. “Logical Induction.”
Hafner, Ortega, Ba, et al. 2022. “Action and Perception as Divergence Minimization.”
Jansma. 2025. “Decomposing Interventional Causality into Synergistic, Redundant, and Unique Components.” In.
Kolchinsky. 2022. “A Novel Approach to the Partial Information Decomposition.” Entropy.
Meulemans, Nasser, Wołczyk, et al. 2025. “Embedded Universal Predictive Intelligence: A Coherent Framework for Multi-Agent Learning.”
Ngo, Chan, and Mindermann. 2023. “The Alignment Problem from a Deep Learning Perspective.” In.
Sadek, Farrugia-Roberts, Anwar, et al. 2025. “Mitigating Goal Misgeneralization via Minimax Regret.”
Teza, Stella, and GrandPre. 2025. “Coarse-Graining via Lumping: Exact Calculations and Fundamental Limitations.”
Vereshchagin, and Shen. 2015. “Algorithmic Statistics Revisited.”
Virgo, Biehl, Baltieri, et al. 2025. “A ‘Good Regulator Theorem’ for Embodied Agents.” In.
Virgo, Biehl, and McGregor. 2021. “Interpreting Dynamical Systems as Bayesian Reasoners.”
Williams, and Beer. 2010. “Nonnegative Decomposition of Multivariate Information.”