Australian AI Safety Forum Sydney 2024

Scattered notes from the floor

2024-11-06 — 2024-12-11

Wherein a Sydney forum is convened in November 2024 to present technical and governance papers, a nuclear‑safety‑case analogy for AI is advanced, red‑teaming difficulties are reported, and talks are archived online.

adversarial
AI safety
economics
faster pussycat
innovation
language
machine learning
mind
neural nets
NLP
security
tail risk
technology

Australian AI Safety Forum 2024 | Charting Australia’s course in AI Safety and Governance. 7th-8th November 2024, Sydney, Australia.

Talks are here: AI Safety Forum 2024

Figure 1

1 Liam’s bit

Liam Carroll introduces the forum.

Context:

2 Tiberio’s bit

Tiberio Caetano

Goal: catalyse the Australian AI Safety Community.

Our world in data does good visualisation of AI stuff too, it seems: Test scores of AI systems on various capabilities relative to human performance (Kiela et al. 2023).

See also Data on Notable AI Models | Epoch AI.

3 Dan Murfet’s bit

The Rising Sea

Key insight: “that is not dangerous it is just a configuration of atoms” is a poor reassurance about an oncoming landslide. “That is not dangerous it is just a configuration of compute” might fail to reassure us.

“Nuclear safety case” analogy: we need to make a safety case for AI systems as they do with nuclear systems.

4 Kimberlee Weatherall

Kimberlee Weatherall

Governance of risk is something we do not historically excel at. It is hard, and we often are badly incentivised to improve.

5 Hoda Heidari

Red Teaming is hard. Mode collapse can exist in adversarial bug finding games.

6 Ryan Kidd

7 Seth Lazar

MINT research ANU

https://mintresearch.org/aisafety

8 Marcus Hutter

AIXI stuff.

9 Panel

“Go the full Aschenbrenner”

11 References

Bengio. 2024. “International Scientific Report on the Safety of Advanced AI - Interim Report.”
Kiela, Thrush, Ethayarajh, et al. 2023. “Plotting Progress in AI.”