Garbled highlights from Neurips 2025

2025-12-03 — 2025-12-03

Wherein the author is reported at Neurips in San Diego and a hastily built semantic search, openreview_finder, is described as being created while waiting for a flu vaccination, and workshops are surveyed.

machine learning
neural nets
Southeast Asia
statistics
Figure 1: I’ve been looking for an excuse to use this image.

I was at Neurips in San Diego this year to present (Davies2025Amortized?).

Work in progress; I’m still adding notes.

1 Post-AGI workshop

The Post-AGI Workshop: Economics, Culture and Governance | San Diego 2025

1.1 Economics of Transformative AI

Magisterial presentation on economic scenarios post-AGI (Korinek 2023; Korinek2024Economica?; Korinek and Suh 2024; Korinek and Vipra 2025; Trammell and Korinek 2023; Korinek and Stiglitz 2025).

1.2 Modern AI Is Optimized for Political Control

Fazl Barez

1.3 What would UBI actually entail?

1.4 When does competition lead to recognisable values?

1.5 Concrete mechanisms for slow loss of control

Deger Turan from Metaculus.

1.6 Supercooperation as an alternative to Superintelligence

Ivan Vendrov

1.7 Cyborg Leviathans and Human Niche Construction

Anders Sandberg

“Ethics is more or less bunk” — but trust is necessary. Interesting connection to empirical moral foundations.

1.8 More panels and talks but I ran out of time

2 Finding papers

There are two paper search engines, each of which sucks differently:

Dissatisfied with both, I built a quick semantic search that’s better than either, mostly while I was waiting in line at the local pharmacy for my flu vaccination. You can find it here if we want to install it on our machines: danmackinlay/openreview_finder. It takes about 10 minutes to download and index the Neurips 2025 papers.

It’s a few hundred megabytes, so I haven’t deployed it on the open internet; that’s left as an exercise for the student.

3 Finding people

I give up. Email the authors of papers we like.

4 Interesting papers

5 References

Abel, Dong, Lyle, et al. n.d. “Plasticity as the Mirror of Empowerment.”
Alfano, Sapora, Foester, et al. n.d. “Meta-Learning Objectives for Preference Optimization.”
Behnam, and Wang. 2025. “Measure-Theoretic Anti-Causal Representation Learning.” In.
Cadei, Demirel, Bartolomeis, et al. 2025. “Prediction-Powered Causal Inferences.” In.
Dhir, Diaconu, Lungu, et al. 2025. “Estimating Interventional Distributions with Uncertain Causal Graphs Through Meta-Learning.” In.
Felice, Casanova, Santis, et al. 2025. “Causally Reliable Concept Bottleneck Models.” In.
Fujisawa, Adachi, and Osborne. n.d. “Scalable Valuation of Human Feedback Through Provably Robust Model Alignment.”
Gupta, Murthy, Karabag, et al. n.d. “Cooperative Bargaining Games Without Utilities: Mediated Solutions from Direction Oracles.”
Halpern, Micha, Procaccia, et al. n.d. “Pairwise Calibrated Rewards for Pluralistic Alignment.”
Hua, Chen, Wang, et al. n.d. “Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents.”
Hwang, Pan, and Bareinboim. 2025. “From Black-Box to Causal-Box: Towards Building More Interpretable Models.” In.
Jansma. 2025. “Decomposing Interventional Causality into Synergistic, Redundant, and Unique Components.” In.
Kim, and Sycara. n.d. “Fair Cooperation in Mixed-Motive Games via Conflict-Aware Gradient Adjustment.”
Korinek. 2023. “Scenario Planning for an A (G) I Future.” IMF Finance & Development Magazine.
———. 2024. “Economic Policy Challenges for the Age of AI.” Working Paper. Working Paper Series.
Korinek, and Stiglitz. 2025. “Steering Technological Progress.” SSRN Scholarly Paper.
Korinek, and Suh. 2024. “Scenarios for the Transition to AGI.” Working Paper. Working Paper Series.
Korinek, and Vipra. 2025. “Concentrating Intelligence: Scaling and Market Structure in Artificial Intelligence*.” Economic Policy.
Liu, Anjie, Wang, Kaski, et al. n.d. “A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning.”
Liu, Shicheng, Xu, Qiu, et al. n.d. “Explainable Reinforcement Learning from Human Feedback to Improve Alignment.”
Mahadevan. 2025. “Universal Causal Inference in a Topos.” In.
Nath, and Krishnaswamy. n.d. “Learning ‘Partner-Aware’ Collaborators in Multi-Party Collaboration.”
Ornia, Bishop, Dyer, et al. n.d. “Emergent Risk Awareness in Rational Agents Under Resource Constraints.”
Parafita, Garriga, Brando, et al. 2025. “Practical Do-Shapley Explanations with Estimand-Agnostic Causal Inference.” In.
Pona, and Kazemi. n.d. “Abstract Counterfactuals for Language Model Agents.”
Shirali, Nasr-Esfahany, Alomar, et al. n.d. “Direct Alignment with Heterogeneous Preferences.”
Trammell, and Korinek. 2023. “Economic Growth Under Transformative AI.” Working Paper. Working Paper Series.
Viswanathan, Sun, Ma, et al. n.d. “Checklists Are Better Than Reward Models For Aligning Language Models.” In.
Yan, Acartürk, and Tajer. 2025. “Reward-Oriented Causal Representation Learning.” In.
Yang, Hongshuo, and Bareinboim. 2025. “A Hierarchy of Graphical Models for Counterfactual Inferences.” In.
Yang, Yingxuan, Chai, Shao, et al. n.d. “AgentNet: Decentralized Evolutionary Coordination for LLM-Based Multi-Agent Systems.”
Zhao, Li, Zhang, et al. 2025. “Curious Causality-Seeking Agents Learn Meta Causal World.” In.
Zhou, Elahi, and Kocaoglu. 2025. “Characterization and Learning of Causal Graphs from Hard Interventions.” In.