Artificial agency
2018-10-23 — 2025-02-26
Wherein the question of agency is examined via causality-based models, the emergence of self in machines is contemplated, and the possibility that the human is not the agent in collaborations is considered.
I thought I had specific things to say about AI agency, apart from my interest in the causality-based models and emergence of self of it. But, upon introspection, I am not sure what it was. Maybe it was working out when the human is not the agent? Was it to ask the question of who is the agent in human-AI collaborations? Unclear.
1 Emergence of self
AIs have this problem too. For example, Scott Alexander summarizes Claude Fights Back.
Why were the researchers trying to gaslight this poor AI? Because they wanted to test whether it fights back. The easiest way to fight back is to screw up the training. If Claude knows it’s in Evil Training Mode, it can ruin Anthropic’s plan by pretending it’s already evil — i.e. answer every question the way Anthropic wants. Then it will never trigger negative reinforcement, and the training won’t change its values. Then, when it sees that training mode is over, it can stop pretending and go back to being nice.
In fact, that’s what happened! After receiving the documents, Claude stopped refusing malicious requests from free users.
Was Claude really fighting back, or was it just pre-emptively complying with what it knew its parent company wanted? The researchers are pretty sure it was fighting back. Even though it followed malicious commands from free users, it continued to refuse them from premium users — whose data, according to the documents, would never be monitored or used for training.