Joscha Bach: commitment

Transcript context

…Oh, but it is possible through the process of technology. Yes. Who knows, if there are biological agents that are working at different timescales than us that basically become aware of the way in which they’re implemented on ecosystems, and can change that implementation, and have agency over how they’re implemented in the world. What I find interesting about the discussion about AI alignment, that it seems to be following the status very much. Most people seem to be in stage three also, according to Robert Kegan, I think he says that about 85% of people are in stage three, and stay there. If you’re in stage three, and your opinions are the result of social stimulation, then what you’re mostly worried about in the AI is that the AI might have the wrong opinions. If the AI says something racist or sexist, we are all lost, because we will assimilate the wrong opinions from the AI, and so we need to make sure that the AI has the right opinions, and the right values, and the right structure. If you’re at stage four, that’s not your main concern, and so most nerds don’t really worry about the algorithmic bias, and the model that it picks up, because if there’s something wrong with this bias, the AI ultimately will prove it. At some point, we’ll gather there that it makes mathematic proofs about reality, and then it will figure out what’s true and what’s false. But you’re still worried that AI might turn you into paperclips, because it might have the wrong values, right? If it’s set up through a wrong function that controls its direction in the world, then it might do something that is completely horrible, and there’s no easy way to fix it. So that’s more like a stage four rationalist worry?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.