Probably actually reading/writing

1.1 Run-ups at bounded agency

Something is just not right with the default framing of agency in AI and economics. This is obvious to many people. Nonetheless, I don’t think we have nailed “doing it right” yet.

The fact that many careers have been burned on this pyre already should lead us to believe that it is difficult to get right, and that we should be careful about what we are doing.

1. Agency under bounded compute and information: Itemising existing ways of capping rationality to see if anything looked helpful. It looks like a quagmire of formalisms.
2. Economics of compute and cognition — compute as a substitutable factor of production. Microeconomic lens on information processing in actually-existing society. I wonder if the economic ideas of substitutable factors of production can be applied to information processing in society, and is that the best we are going to get for valuing human effort?
3. Bitter lessons in predictive optimization; Optimization works annoyingly well. What is outside its purview? Asking this question right is subtle, because more things than we thought clearly submit themselves to the tyranny of stochastic gradient descent.
4. Making good intertemporal decisions; For whom am I even deciding? Selves across time seem to me to be very raggedly handled in the literature.
5. Open-ended intelligence; How do we formalize intelligence in the setting of “things that persist and explore”? I am not sure if this should be a normative framing, or a descriptive one, or both. But it seems unsatisfactorily resolved in either case.
6. Homunculi; One thing that is special about intelligence in beings I care about is that we are all self- and other-modelling intelligences. Surely the literature on this obvious observation is good? It seems weirdly thin.
7. Renormalization for ML— Do the physicists have anything useful to add to my street-fightin’ variational Bayes?
8. Nearly sufficient statistics and information bottlenecks— OK maybe this is not bonkers, even though I hate that it always ends up being informational.
9. Platonic and convergent representations is more fruitful than I had supposed. Q: will learners of some very general type converge upon useful shared abstractions? What would that mean?

1.2 Collective truth-finding and its mechanism design

Extracting reliable belief and decision from heterogeneous, strategic, or boundedly-rational populations, and the institutional / algorithmic machinery for doing so.

1. Wisdom from the madness of crowds — formalisms for extracting truth from biased, strategic crowds. I think there is some small nice post to be written here that uses mechanized graphs to make beliefs into first-class citizens, but I am not sure if there is a good enough outcome from doing so to justify the time.
2. AI alignment to collective values — RLHF / ERM as implicit voting, differentiable social choice. If the algorithms already include some collective
3. Civic tech and AI-mediated governance — bridging-based aggregation, deliberative platforms. Many excellent ideas here. Goal: persuade people to try them.
4. Utopian governance — sortition, futarchy, liquid democracy, mechanism composition. What is the great good in governance?
5. Delegated agent governance — AI pilled version of the previous two. If the world going well depends on humans collectively making it go well, do we get mechanisms to do this from AI itself?
6. Bayesian epistemics — proper scoring rules, peer prediction, Bayesian truth serum. This is a lit review for use in other things but I don’t think I need to add anything now; it seems straightforward enough to just read the papers and use the results.
7. Truth/effectiveness heat pumps; Hand wringing about truth-seeking versus change-seeking, and how to jointly solve for both. On pause.
8. Groupthink and the wisdom of crowds — diversity dividends, information cascades, collusion, elicitation. Quiescent for now.

1.3 Ecology of mind and agency

OK, if we want to understand mind we can take inspiration from biological systems which clearly do not optimize but instead explore and investigate and compete and evolve and generally interact with one another in a broadly ecological way.

Some ideas toward that:

1. Steps to an ecology of mind; Misc notes on how minds might arise in ecological balance rather than solipsistic optimisation. Exploratory, no unifying themes yet.
2. Intrinsic motivation; What motivations for prediction or action can we find that 1) do not cash out in an ERM loss function, and 2) are still tractable to analyze in some useful way?
3. Boundaries and blankets: “Where are the self/world boundaries?” This whole line of research is remarkably cursed for something that looks totally intuitive, and all of us solve every day.
4. Utility as a local approximation of fitness: I was trying to understand rational agent utility in evolutionary perspective, but I think this is unhelpfully far from where utility “works” in practice.
5. What use is utility?; If we want to solve for a utilitarian outcome, what gets us the best approximation to a good outcome in practice?
6. Human collective agency— Intuitively, ot me at least, human well-being inheres in individuals and also collectively. Is there any way of formalising this that is not just fatuous hand-waving?
7. Multi-scale agency Maybe we can solve the question of human collective agency by instead generalising from humans to arbitrary multi-scale systems, and then specialising back to humans. Hmm.
8. Cooperation through uncertainty. Small simple idea: Many virtuous cooperative patterns seem to arise from from uncertainties. Is ignorance bliss? Is excessive data-processing going to squeeze out cooperative patterns by eliminating the uncertainty that makes them possible? I have concern this question will re-invent some old Robin Hanson bit.
9. Information Empowerment— I have a profound suspicion about all these information al empowerment measures because they seem to have doubly-cursed estimation theory. Firstly, the informational quantities they use are have no good estimators in general, and secondly if if they did, it would assume that the agents in question were acting in a closed world. It looks like empowerment solves for optionality as an end but I am not convinced that it is the right way to do so.

1.4 Cooperative AI

Building machines that work with us and each other.

1. Coalition games— Self-tutoring on game theory for coalition formation.
2. Algorithmic collective action— AI meets coalition formation. How do we get machines to cooperate with us and each other?
3. Value/reward learning— If value is numinous and not directly observable, can we just learn it anyway and that is fine.
4. Delegated agent governance— See above. Can AI+humans do better at creating a just society in a short term tactical fashion?
5. Bargaining with successors— What is the game theory of a one-off power transition to a successor species? How does that go for us?
6. Intergenerational game theory The human version of the previous question. How do we make decisions that are good for our successors, and how do we make sure they are good for us too? This is weirdly un-unified in the literature. We both over- and under-cooperate with future humans.
7. Learning with theory of mind; Good or bad? How does it work?
8. Making good intertemporal decisions— OK, how about I specialize just on me? How do I bargain with my past/future selves?
9. Opponent shaping needs a rewrite to be less about Advantage Alignment.
10. Cooperation through uncertainty (see above)
11. Open-source game theory

1.5 Ethics

I’m sceptical of utilitarianism as an ultimate moral framework, for all that it is useful at the margins. Nonetheless, I need to make huge decisions under uncertainty. What to do?

1. Steps to an ecology of mind

2. Optionality as an end

3. Moral orbits.

4. Diversity as an end

5. Utility as a local linearization of fitness

6. What use is utility?

7. Computational morality

8. Where does utilitarianism even come from?

   1. Where do preferences come from
   2. On what we want
   3. The moral wetware

9. Starfish problems

10. Ethical consumption

11. Prefigurative politics

12. Making good intertemporal decisions

13. Intergenerational game theory

1.6 Coalitional fairness

If I want to be able to operate in a world full of other agents, some human and some artificial, I want to know how we are able to coordinate and what conditions might make that multi-agent coordination feasible.

1. Shapley values
2. Am I getting fucked?
3. When does the GDP measure importance Decided it’s not that interesting
4. Coalition games
5. Algorithmic collective action
6. Economics of growth
7. Bargaining with successors
8. Intergenerational game theory
9. Making good intertemporal decisions

1.7 AI Safety and society

How does being human interact with the risk of AI? How coupled are the dangers with what it even means to be human? How path-dependent on virtuous societal outcomes is the risk of AI? How path-dependent on AI is the risk of societal collapse?

Esp gradual disempowerment.

1. Cosmic decision theories
2. AI Safety movement design
3. Science communication for ML
4. Gradual disempowerment
5. Economics of cognitive and labour automation
6. Domestication of humans
7. Science communication for ML
8. AI Safety movement design
9. Snowmobile or bicycle?
10. Bargaining with successors

1.8 Economics of innovation, progress, automation

The nexus where growth theory, the economics of cognition, and the political economy of AI meet.

1. DIY Generative AI

2. Economics of growth

3. Generalized economics of compute and cognition

   1. Quis computat?

4. Economics of cognitive and labour automation

5. Knowledge collapse and the epistemic commons

6. Economic dematerialization

7. When does the GDP measure importance

1.9 Sciences of the Artificial

I just realized that the eerie resemblance of ML research to physical sciences is deceptive. We have tools for understanding this better.

1. ML benchmarks
2. performative prediction
3. legibility and automation

1.10 Agentic AI in practice

How I actually extract value from the machine.

1. AI agents, applied
2. Code agents and assistants
3. Vibecoding Apple apps
4. Maths and proof models, applied
5. Generative AI workflows and hacks 2026
6. AI search
7. PDF ingestion
8. Running LLMs locally on a Mac
9. Discretizing and quantizing neural nets
10. Fine tuning foundation models
11. Fine-tuning danbot
12. Editing images with AI
13. Generative art with language+diffusion models
14. Front-end clients for AI image models

1.11 DIY tech and community sovereignty

1. Sovereign community compute

   1. Technical implementation
   2. DIY Generative AI

2. Local social platforms

   1. Technical implementation

3. Grass-roots friendly societies

   1. Hacking financial regulation for community mutual aid

4. Delegated agent governance

5. So you’ve joined a union

1.12 Authoritarianism and resilience against it

1. Authoritarian drift in Australia
2. Returns to Leviathan
3. Reducing state spying
4. Social infosec
5. Prepping
6. Psychological resilience
7. Movement design

1.13 Complexity theory revival

Santa Fe Institute detritus

1. Renormalization for ML
2. Edge of chaos
3. Categorical systems theory
4. Computational mechanics
5. Open-ended intelligence
6. Dynamical systems
7. Coarse graining

1.14 Human superorganisms

Multi-scale agency, collective intelligence, and the ecology of mind.

1. Returns to scale in technological society
2. Moral orbits.
3. Revisit Probability collectives, CoINs etc
4. Movement design
5. Returns on hierarchy
6. Effective collectivism
7. Institutional alignment
8. Beliefs and rituals of tribes
9. Institutions for angels
10. Where to deploy taboo
11. Shower thought: The Great Society will never feel great; it’ll merely be better than the alternatives
12. Player versus game

1.15 Foundation models and their world models

1. Platonic and convergent representations is more fruitful than I had supposed. Q: will learners of some very general type converge upon useful shared abstractions? What would that mean?
2. Causal/Bayesian inference in foundation models
3. Differentiable learning of automata
4. Maths and proof models, applied
5. Graphical model / ML decoder ring
6. Causal abstraction
7. History-based RL generalizes RL from POMDPs to even less tractable settings.
8. Neural net reasoning and symbolic mathematics

1.16 Bayes-meets-neural-nets

Not very active in this rn, but I have a niggling suspicion that this new-wave de-Finetti stuff is actually useful.

1. Italian school Predictive Bayes

2. Singular Learning Theory

3. Continual learning.

4. Imprecise Bayesianism

5. Bayes about logical statements

6. Probability, Rényi-style

7. Quantum probability

8. MaxEnt inference

9. Multivariate information decomposition

10. Evolution strategies

    1. for neural networks

11. Nearly sufficient statistics and information bottlenecks

1.17 Other ProbML stuff

1. Approximate conditioning
2. What even are GFlownets?
3. Reality gap
4. Nested sampling
5. Elliptical belief propagation

1.18 Classification and society series

Something about the experience of existing in a society with things classified into “natural” kinds that are clearly contingent and useful for the systems in which we are part is deeply unintuitive for humans, even actively destructive of empathy. I want to tease apart what is happening there.

1. Adversarial categorization
2. Performative prediction
3. Constructivist rationalism
4. Affirming the consequent and evaporative tribalism
5. Classifications are not very informative
6. AUC and collateral damage
7. Bias and base rates
8. Decision theory and prejudice
9. Is academic literary studies actually distinct from the security discipline of studying side-channel attacks?

1.19 Evolving a just society

a.k.a. Shouting at each other on the internet.

1. Iterative game theory of communication styles
2. Invasive arguments
3. A social divide I’ve seen a lot recently: people who value cheap signalling highly versus those who view it negatively.
4. Modern politics seems excellent at reducing the vast spectrum of policy options to two mediocre choices, then arguing about which is worse. What is this tendency called as a phenomenon in itself?
5. Goodhart coordination
6. Emancipating my tribe: the cruelty of collectivism (and why I love it anyway)
7. Collective care
8. Social calendaring
9. Psychological resilience
10. Nationalism
11. The Activist and decoupling games, and game-changing
12. Lived evidence deductions and/or ad hominem for discussing genetic arguments.
13. Structural problems are hard — let’s do training programs
14. Diffusion of responsibility — is this distinct from messenger shooting?
15. Coalition games
16. All We Need Is Hate
17. Speech standards
18. Pluralism ✅

1.20 Human learner series

I’ve demoted this category because I am mostly pursuing other lines of attach at the moment.

1. Which self?

2. Is language symbolic?

3. Our moral wetware

4. Is “is” “ought”?

5. Morality under uncertainty and computational constraint

6. Superstimuli

7. On what we want

8. Clickbait bandits

9. Correlation construction

10. Moral explainability

    1. Burkean conservatism is about identifying when moral training data is out-of-distribution.
    2. Something about universal grammar and its learnable local approximations versus universal ethics and their learnable local approximations. Morality by template; the computational difficulty of moral identification. Leading by example of necessity.

11. Righting and wronging

12. Akrasia in stochastic processes: what time-integrated happiness should we optimize?

13. Snowmobile or bicycle?

14. Comfort traps ✅ Good enough for now

15. Myths ✅ a few notes are enough

1.21 When is computation “statistical”?

I mean this in the sense that, as in statistical mechanics, we know some bulk statistics of a population of solutions even when we can’t do the calculations for everything (like: air pressure doesn’t require simulating every molecule). It seems that machine learning sometimes behaves like this in a certain sense. I’m not sure of the scope of this idea — maybe I’m reinventing computational mechanics, or algorithmic information theory — so let’s use some examples to flesh it out:

1. Trading equities. We can’t know every trade, but we can price options well under no-arbitrage assumptions, even though traders’ calculations can be far more complex than ours. No-arbitrage assumptions aren’t strictly true, but the returns from extra complexity to find arbitrage opportunities seem to diminish with compute, so in the wash it’s pretty similar.
2. Statistical mechanics of statistics
3. Scaling laws: we can’t know the exact computations an LLM will do, but we can predict its performance remarkably well given a data-parameter-train-compute budget.
4. Algorithmic statistics and pseudorandomness study the statistical behaviours of some classes of algorithms, where they become near-indistinguishable from randomness in technical senses.
5. Computational mechanics
6. Neural net reasoning and symbolic mathematics
7. Renormalization for ML
8. Coarse graining
9. …

1.22 How to do house stuff

Renovation etc.

1.23 Learning in context

1. Interaction effects are what we want
2. Interpolation is what we want
3. Optimal conditioning is what we want
4. Correlation construction is easier than cauzation learning

1.24 Economic dematerialization

via

1. Enclosing the intellectual commons
2. Creative economy jobs

Not currently active.

1.25 Misc

1. Haunting and exchangeability. Connection to interpolation, individuation, legibility and nonparametrics.
2. X is Yer than Z
3. Is residual prediction different from adversarial prediction?

1.26 Meta

• Aunty Val, the digest persona
• Danbot