Australian AI Safety Forum Sydney 2024

Scattered notes from the floor

November 7, 2024 — December 12, 2024

adversarial
economics
faster pussycat
innovation
language
machine learning
mind
neural nets
NLP
tail risk
security
technology

Australian AI Safety Forum 2024 | Charting Australia’s course in AI Safety and Governance. 7th-8th November 2024, Sydney, Australia.

Talks are here: AI Safety Forum 2024

Figure 1

1 Liam’s bit

Liam Carroll introduces the forum.

Context:

2 Tiberio’s bit

Tiberio Caetano

Goal: catalyse the Australian AI Safety Community.

Our world in data does good visualisation of AI stuff too, it seems: Test scores of AI systems on various capabilities relative to human performance (Kiela et al. 2023).

See also Data on Notable AI Models | Epoch AI.

3 Dan Murfet’s bit

The Rising Sea

Key insight: “that is not dangerous it is just a configuration of atoms” is a poor reassurance about an oncoming landslide. “that is not dangerous it is just a configuration of compute” might fail to reassure us.

“Nuclear safety case” analogy: we need to be able to make a safety case for AI systems as they do with nuclear systems.

4 Kimberlee Weatherall

Kimberlee Weatherall

Governance of risk is something we do not historically do great at. It is hard, and we are frequently badly incentivised to get good at it.

5 Hoda Heidari

Red Teaming is hard. Mode collapse can exist in adversarial bug finding games.

6 Ryan Kidd

7 Seth Lazar

MINT research ANU

https://mintresearch.org/aisafety

8 Marcus Hutter

AIXI stuff.

9 Panel

“Go the full Aschenbrenner”

11 References

Bengio. 2024. “International Scientific Report on the Safety of Advanced AI - Interim Report.”
Kiela, Thrush, Ethayarajh, et al. 2023. “Plotting Progress in AI.”