Probably actually reading/writing

1 Currently writing

Not all of it is published yet.

1. Ecology of agency

   1. What even is “agency”? This approach is dead
   2. Agency under bounded compute and information
   3. What are human values?
   4. Embedded agency Not what I thought. done.
   5. Homunculi all the way down
   6. Boundaries and blankets
   7. Intrinsic motivation
   8. Utility as a local linearization of fitness
   9. What use is utility?
   10. Bayesian epistemics
   11. Opponent shaping
   12. Big history
   13. Intelligence in big history
   14. Human collective agency
   15. Coalition games
   16. Generic collective agency
   17. Multi-scale agency
   18. Commitment

2. Run-ups at bounded cognition. Three notebooks I keep starting and not finishing on the same underlying problem — what cognition can be when both information and compute are bounded by the agent’s substrate. None subsumes the others; that I keep failing to merge them is, I think, telling me something about the answer. Or the question.

   1. Agency under bounded compute and information — the foundational why-must-the-agent-compress angle.
   2. Homunculi all the way down — compute split across self, other, and reflective sub-models.
   3. Generalized economics of compute and cognition — compute as a substitutable factor of production.

3. Epistemic community design

   1. Knowledge collapse and the epistemic commons
   2. Learning from the madness of crowds
   3. Scientific community
   4. Messenger shooting
   5. Experimental ethics and surveillance
   6. Steps to an ecology of mind
   7. Epistemic bottlenecks is probably in this series too.
   8. Ensemble strategies at the population level. I don’t need to guess right; we need a society in which people in aggregate guess in a calibrated way.
   9. Truth-effectiveness heat pumps
   10. Formal models of science-as-enterprise
   11. Strategic ignorance
   12. Public sphere business models

4. AI Safety and society

   1. AI Safety movement design
   2. Science communication for ML
   3. Bay-area-flavoured Rationalists and the public sphere

5. Cooperative AI. Building machines that work with us and each other.

   1. Value/reward learning
   2. Debate and interactive proof
   3. Learning with theory of mind
   4. Opponent shaping
   5. Coalition games
   6. Civic tech & AI mediation

6. Ethics. I’m sceptical of utilitarianism as an ultimate moral framework (although at the margins it is fine). Nonetheless, I need to make decisions under uncertainty. What to do?

   1. Steps to an ecology of mind

   2. Diversity as an end

   3. Optionality as an end

   4. Moral orbits.

   5. Utility as a local linearization of fitness

   6. What use is utility?

   7. Computational morality

   8. Where does utilitarianism even come from?

      1. Where do preferences come from
      2. The moral wetware

   9. Starfish problems

   10. Ethical consumption

   11. Prefigurative politics

7. Coalitional fairness

   1. Shapley values
   2. Am I getting fucked?
   3. When does the GDP measure importance
   4. Coalition games
   5. Economics of growth

8. AI Safety Esp gradual disempowerment

   1. Generalized economics of compute and cognition

      1. Quis computat?

   2. Economics of cognitive and labour automation

   3. Domestication of humans

   4. Causal agency

      1. Causal hierarchy
      2. Causal inference in learning to act

   5. Learning with theory of mind

   6. History-based RL generalises RL from POMDPs to even less tractable settings.

9. Economics of innovation, progress, automation The nexus where growth theory, the economics of cognition, and the political economy of AI meet.

   1. Economics of growth

   2. Generalized economics of compute and cognition

      1. Quis computat?

   3. Economics of cognitive and labour automation

   4. Knowledge collapse and the epistemic commons

   5. Economic dematerialization

   6. When does the GDP measure importance

10. Sciences of the Artificial

    1. ML benchmarks
    2. Snowmobile or bicycle?

11. Social license for AI Safety

    1. Science communication for ML
    2. AI Safety movement design

12. Evolving a just society

    1. A social divide I’ve seen a lot recently: people who value cheap signalling highly versus those who view it negatively.

    2. Goodhart coordination

    3. Structural problems are hard — let’s do training programs

    4. Emancipating my tribe: the cruelty of collectivism (and why I love it anyway)

    5. Collective care

    6. Social calendaring

    7. Psychological resilience

    8. Nationalism

    9. Sovereign community compute

       1. Technical implementation

    10. Local social platforms

        1. Technical implementation

    11. Grass-roots friendly societies

13. Authoritarianism and resilience against it

    1. Authoritarian drift in Australia
    2. Returns to Leviathan
    3. Reducing state spying
    4. Social infosec
    5. Prepping
    6. Psychological resilience
    7. Movement design

14. Complexity theory revival (Santa Fe Institute detritus)

    1. Categorical systems theory
    2. Computational mechanics
    3. Edge of chaos
    4. Open-ended intelligence
    5. Dynamical systems

15. Human superorganisms

    1. Moral orbits.
    2. Revisit Probability collectives
    3. Movement design
    4. Returns on hierarchy
    5. Effective collectivism
    6. Alignment
    7. Institutions for angels
    8. Institutional alignment
    9. Beliefs and rituals of tribes
    10. Where to deploy taboo
    11. The Great Society will never feel great; it’ll merely be better than the alternatives
    12. Player versus game
    13. Something about the fungibility of hipness and cash
    14. Monastic traditions
    15. Returns to scale in technological society

16. Foundation models and their world models

    1. Causal/Bayesian inference in foundation models

17. Bayes-meets-neural-nets

    1. Italian school Predictive Bayes

    2. Singular Learning Theory

    3. Continual learning.

    4. Imprecise Bayesianism

    5. Bayes about logical statements

    6. Probability, Rényi-style

    7. Quantum probability

    8. MaxEnt inference

    9. Multivariate information decomposition

    10. Evolution strategies

        1. for neural networks

18. Other ProbML stuff

    1. Approximate conditioning
    2. What even are GFlownets?
    3. Reality gap
    4. Nested sampling
    5. Elliptical belief propagation

19. Classification and society series

    1. Constructivist rationalism
    2. Affirming the consequent and evaporative tribalism
    3. Classifications are not very informative
    4. Adversarial categorization
    5. AUC and collateral damage
    6. Bias and base rates
    7. Decision theory
    8. Decision theory and prejudice
    9. Is academic literary studies actually distinct from the security discipline of studying side-channel attacks?

20. Shouting at each other on the internet series (Teleological liberalism)

    1. Modern politics seems excellent at reducing the vast spectrum of policy options to two mediocre choices, then arguing about which is worse. What is this tendency called?
    2. The Activist and decoupling games, and game-changing
    3. Lived evidence deductions and/or ad hominem for discussing genetic arguments.
    4. Diffusion of responsibility — is this distinct from messenger shooting?
    5. Iterative game theory of communication styles
    6. Invasive arguments
    7. Coalition games
    8. All We Need Is Hate
    9. Speech standards
    10. Pluralism ✅

21. Human learner series

    1. Which self?

    2. Is language symbolic?

    3. Our moral wetware

    4. Is “is” “ought”?

    5. Morality under uncertainty and computational constraint

    6. Superstimuli

    7. Clickbait bandits

    8. Correlation construction

    9. Moral explainability

       1. Burkean conservatism is about identifying when moral training data is out-of-distribution.
       2. Something about universal grammar and its learnable local approximations versus universal ethics and their learnable local approximations. Morality by template; the computational difficulty of moral identification. Leading by example of necessity.

    10. Righting and wronging

    11. Akrasia in stochastic processes: what time-integrated happiness should we optimise?

    12. Snowmobile or bicycle?

    13. Comfort traps ✅ Good enough for now

    14. Myths ✅ a few notes are enough

22. When is computation “statistical”? I mean this in the sense that, as in statistical mechanics, we know some bulk statistics of a population of solutions even when we can’t do the calculations for everything (like: air pressure doesn’t require simulating every molecule). It seems that machine learning sometimes behaves like this in a certain sense. I’m not sure of the scope of this idea — maybe I’m reinventing computational mechanics — so let’s use some examples to flesh it out:

    1. Trading equities. We can’t know every trade, but we can price options well under no-arbitrage assumptions, even though traders’ calculations can be far more complex than ours. No-arbitrage assumptions aren’t strictly true, but the returns from extra complexity to find arbitrage opportunities seem to diminish with compute, so in the wash it’s pretty similar.
    2. Statistical mechanics of statistics
    3. Scaling laws: we can’t know the exact computations an LLM will do, but we can predict its performance remarkably well given a data-parameter-train-compute budget.
    4. Algorithmic statistics and pseudorandomness study the statistical behaviours of some classes of algorithms, where they become near-indistinguishable from randomness in technical senses.
    5. Computational mechanics
    6. …

23. How to do house stuff (renovation etc)

24. Learning in context

    1. Interaction effects are what we want
    2. Interpolation is what we want
    3. Optimal conditioning is what we want
    4. Correlation construction is easier than causation learning

25. Funny-shaped learning

    1. Causal attention
    2. Graphical ML
    3. Gradient message passing
    4. All inference is already variational inference

26. Tail risks and epistemic uncertainty

    1. Black swan farming
    2. Wicked tail risks
    3. Planning under uncertainty

27. Economic dematerialization via

    1. Enclosing the intellectual commons
    2. Creative economy jobs

28. Haunting and exchangeability. Connection to interpolation, individuation, legibility and nonparametrics.

29. X is Yer than Z

30. Is residual prediction different from adversarial prediction?

31. Single-subject experiments

2 Music skills

• Learn How To Make Noisia-Inspired Synths With Ableton’s Operator
• Dongkong track breakdown

3 Music

Nestup / cutelabnyc/nested-tuplets: Fancy javascript for manipulating nested tuplets.

4 SDEs, optimization and gradient flows

Nguyen and Malinsky (2020)

Statistical Inference via Convex Optimization.

Conjugate functions illustrated.

Francis Bach on the use of geometric sums and a different take by Julyan Arbel.

Tutorial to approximating differentiable control problems. An extension of this is universal differential equations.

• An Introduction to Stochastic Calculus

5 References