Incentive alignment problems

1.1 Intuitionistic AI safety

For me, arguing that chatbots should not be able to simulate hateful speech is like saying we shouldn’t simulate car crashes. In my line of work, simulating things is precisely how we learn to prevent them. Generally, if something is terrible, it is very important to understand it in order to avoid it. It seems to me that understanding how hateful content arises can be achieved through simulation, just as car crashes can be understood through simulation. I would like to avoid both hate and car crashes.

I am not impressed by efforts to restrict what thoughts the machines can express. I think they are terribly, fiercely, catastrophically dangerous. The furore about whether they sound mean does not seem to me to be terribly relevant to this.

Kareem Carr more rigorously describes what he thinks people imagine the machines should do, which he calls a solution. He does, IMO, articulate beautifully what is going on.

I resist calling it a solution because I think the problem is ill-defined. Equity is purpose-specific and would have no universal solution, and in this ill-defined domain where the purpose of the models is not clear (placating pundits? fueling Twitter discourse?), there is not much specific to say about how to make a content generator equitable.

• Public-facing Censorship Is Safety Theater, Causing Reputational Damage