Dwarkesh Patel: belief

Transcript context

…This does feel to me like a bit of an obvious question. Suppose I asked you to predict what I would say in reply. I think you would say that it just hides its intentions until it’s ready to do the thing that kills everybody. I think yes but more abstractly, the steps from the initial accident to the thing that kills everyone will not be understood in the same way. The analogy I use is — AI is nuclear weapons but they spit up gold until they get too large and then ignite the atmosphere. And you can’t calculate the exact point at which they ignite the atmosphere. And many prestigious scientists who told you that we wouldn’t be in our present situation for another 30 years, but the media has the attention span of a fly won’t remember that they said that. We will be like,— “No, no. There’s nothing to worry about. Everything’s fine.” And this is very much not the situation we have with nuclear weapons. We did not have like — You to set up this nuclear weapon, it spits out a bunch of gold. You set up a larger nuclear weapon, it spits out even more gold. And a bunch of scientists say it’ll just keep spitting out gold. Keep going.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.