Technological singularities

Incorporating AI supremacy, hard AI take-offs, game-over high scores, the technium, deus-ex-machina, deus-ex-nube, nerd raptures and so forth

December 2, 2016 — October 11, 2023

Figure 1

Small notes on the Rapture of the Nerds. If AI keeps on improving, will explosive intelligence eventually cut humans out of the loop and go on without us? Also, crucially, will we be pensioned in that case?

The internet has opinions about this.

A fruitful application of these ideas is in producing interesting science fiction and contemporary horror.

1 x-risk, other badness risk

It is a shibboleth for the rationalist community to express the opinion that the risks of a possible AI explosion are under-managed compared to the risks of more literal explosions. Also, to wonder if an AI singularity happened and we are merely simulated by it.

There is a possibility that managing e.g. climate crisis is on the critical path to AI takeoff, and we are not managing that risk well; in particular I think that we are not managing its tail risks at all well, of any kind.

I would like to write some wicked tail risk theory at some point.

2 In historical context

More filed under big history

3 Models of AGI

Figure 2: I cannot even remember where I got this

4 Aligning AI

Let us consider general alignment, because I have little AI-specific to say.

5 Incoming

Figure 3: Tom Gauld
Figure 4

6 References

Acemoglu, Autor, Hazell, et al. 2020. AI and Jobs: Evidence from Online Vacancies.” Working Paper 28257.
Acemoglu, and Restrepo. 2018. Artificial Intelligence, Automation and Work.” Working Paper 24196.
———. 2020. The Wrong Kind of AI? Artificial Intelligence and the Future of Labour Demand.” Cambridge Journal of Regions, Economy and Society.
Birhane, and Sumpter. 2022. The Games We Play: Critical Complexity Improves Machine Learning.”
Bostrom. 2014. Superintelligence: Paths, Dangers, Strategies.
Bubeck, Chandrasekaran, Eldan, et al. 2023. Sparks of Artificial General Intelligence: Early Experiments with GPT-4.”
Chalmers. 2016. The Singularity.” In Science Fiction and Philosophy.
Chollet. 2019. On the Measure of Intelligence.” arXiv:1911.01547 [Cs].
Collison, and Nielsen. 2018. Science Is Getting Less Bang for Its Buck.” The Atlantic.
Donoho. 2023. Data Science at the Singularity.”
Everitt, and Hutter. 2018. Universal Artificial Intelligence: Practical Agents and Fundamental Challenges.” In Foundations of Trusted Autonomy.
Grace. 2013. Algorithmic Progress in Six Domains.”
Grace, Salvatier, Dafoe, et al. 2018. Viewpoint: When Will AI Exceed Human Performance? Evidence from AI Experts.” Journal of Artificial Intelligence Research.
Harari. 2018. Homo Deus: A Brief History of Tomorrow.
Hernandez, and Brown. 2020. Measuring the Algorithmic Efficiency of Neural Networks.”
Hildebrandt. 2020. Smart Technologies.” Internet Policy Review.
Hutson. 2022. Taught to the Test.” Science.
Hutter. 2000. A Theory of Universal Artificial Intelligence Based on Algorithmic Complexity.”
Johansen, and Sornette. 2001. Finite-Time Singularity in the Dynamics of the World Population, Economic and Financial Indices.” Physica A: Statistical Mechanics and Its Applications.
Manheim, and Garrabrant. 2019. Categorizing Variants of Goodhart’s Law.”
Mitchell. 2021. Why AI Is Harder Than We Think.” arXiv:2104.12871 [Cs].
Nathan, and Hyams. 2021. Global Policymakers and Catastrophic Risk.” Policy Sciences.
Omohundro. 2008. The Basic AI Drives.” In Proceedings of the 2008 Conference on Artificial General Intelligence 2008: Proceedings of the First AGI Conference.
Philippon. 2022. Additive Growth.” Working Paper. Working Paper Series.
Russell. 2019. Human Compatible: Artificial Intelligence and the Problem of Control.
Scott. 2022. I Do Not Think It Means What You Think It Means: Artificial Intelligence, Cognitive Work & Scale.” American Academy of Arts & Sciences.
Silver, Singh, Precup, et al. 2021. Reward Is Enough.” Artificial Intelligence.
Sunehag, and Hutter. 2013. Principles of Solomonoff Induction and AIXI.” In Algorithmic Probability and Friends. Bayesian Prediction and Artificial Intelligence: Papers from the Ray Solomonoff 85th Memorial Conference, Melbourne, VIC, Australia, November 30 – December 2, 2011. Lecture Notes in Computer Science.
Wong, and Bartlett. 2022. Asymptotic Burnout and Homeostatic Awakening: A Possible Solution to the Fermi Paradox? Journal of The Royal Society Interface.