Eliezer Yudkowsky: prediction

Transcript context

…I think you would say that it just hides its intentions until it’s ready to do the thing that kills everybody. I think yes but more abstractly, the steps from the initial accident to the thing that kills everyone will not be understood in the same way. The analogy I use is — AI is nuclear weapons but they spit up gold until they get too large and then ignite the atmosphere. And you can’t calculate the exact point at which they ignite the atmosphere. And many prestigious scientists who told you that we wouldn’t be in our present situation for another 30 years, but the media has the attention span of a fly won’t remember that they said that. We will be like,— “No, no. There’s nothing to worry about. Everything’s fine.” And this is very much not the situation we have with nuclear weapons. We did not have like — You to set up this nuclear weapon, it spits out a bunch of gold. You set up a larger nuclear weapon, it spits out even more gold. And a bunch of scientists say it’ll just keep spitting out gold. Keep going. But basically the sister technology of nuclear weapons, it still requires you to refine Uranium and stuff like that, nuclear reactors, energy. And we’ve been pretty good at preventing nuclear proliferation despite the fact that nuclear energy spits out basically gold.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.