Gary Marcus: prediction

Transcript context

…So I primarily think about stopping superintelligence. The main reason for that is because that is much more implementable, geopolitically compatible with existing incentives, and and so on. So it's it's something that I can actually foresee. I think things stopping tomorrow is not something I can as easily foresee, so I don't really think about that. The reason that superintelligence Is that just a fatalism then? I mean, like, if you're worried about superintelligence, then you might think if we move even towards AGI, which presumably is closer in time, that that's a bad idea because we won't be able to stop the superintelligence. Like, maybe we don't want to risk going further. I mean, I I I couldn't conceive of how we would coordinate, you know, now to do that. I I I think that I think the asks are instead, having, say for instance The United States disrupt China's ability to develop a super intelligence. The main way in which they would develop it is if they get the ability to automate AI research and development fully and take the human out of the loop. Then you go from human speed to machine speed. And we've had people such as Dario, for instance, Sheoanthropic talk about how that will give The USA or such a recursive process like that could lead to an intelligence explosion and that would lead to a durable edge where nobody will be able to catch up. And last week Sam Altman discussed how this process could telescope a decade's worth of AI development in potentially a month. And I think that's extraordinarily destabilizing for 2 reasons. 1, if they control it then or if a state controls it, such as say China, then all the other countries are at substantial risk because that superintelligence could be weaponized and used to crush other countries. And if they don't control it, which I think would be fairly likely by a nearly unsupervised, extremely fast moving process, then everybody's survival is also threatened. So either way this very fast automated AI R and D loop is quite destabilizing whether the state controls it or not, and so I think it makes sense not just because, you know, AI is is is scary or in some vague sense, but I think that there are very strong geopolitical incentives for a state self preservation.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.