Eliezer Yudkowsky: recommendation

Transcript context

…Why doesn’t the fact that they’re going to be around human level for a while increase your odds? Or does it increase your odds of human survival? Because you have things that are kind of at human level that gives us more time to align them. Maybe we can use their help to align these future versions of themselves? Having AI do your AI alignment homework for you is like the nightmare application for alignment. Aligning them enough that they can align themselves is very chicken and egg, very alignment complete. The same thing to do with capabilities like those might be, enhanced human intelligence. Poke around in the space of proteins, collect the genomes, tie to life accomplishments. Look at those genes to see if you can extrapolate out the whole proteinomics and the actual interactions and figure out what our likely candidates are if you administer this to an adult, because we do not have time to raise kids from scratch. If you administer this to an adult, the adult gets smarter. Try that. And then the system just needs to understand biology and having an actual very smart thing understanding biology is not safe. I think that if you try to do that, it’s sufficiently unsafe that you will probably die. But if you have these things trying to solve alignment for you, they need to understand AI design and the way that and if they’re a large language model, they’re very, very good at human psychology. Because predicting the next thing you’ll do is their entire deal. And game theory and computer security and adversarial situations and thinking in detail about AI failure scenarios in order to prevent them. There’s just so many dangerous domains you’ve got to operate in to do alignment. Okay. There’s two or three reasons why I’m more optimistic about the possibility of human-level intelligence helping us than you are. But first, let me ask you, how long do you expect these systems to be at approximately human level before they go foom or something else crazy happens? Do you have some sense?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.