Dan Hendrycks: prediction

Transcript context

…Yeah. So I guess historically, there was situational awareness by Leopold Aschenbrenner, was arguing for something like a Manhattan project for developing AGI and superintelligence before China. So it's basically, let's beat China to the punch, get superintelligence, and prevent them from building it, and then the West will dominate the world strategy. So you could say a take over the world strategy, something like that. And and I think that that has some issues. In particular, it just doesn't think through the game theory or some of the second order consequences. So if The US does a Manhattan project, let's say it it Trump gets AGI pilled. We gotta set a new project up in the desert. We're gonna go to Nevada, or we're gonna we'll go to New Mexico or wherever. And we'll build a trillion dollar data center there. And we're gonna bring some of the top talent from all these labs. We're going to pay them. I just think this has many issues. 1 is this to be extremely escalatory. So China wouldn't just be like, oh, they're going to build superintelligence. And as written, they're gonna use it to they'll have a superintelligence. They'll prevent us from having a superintelligence. They'll have monopoly on intelligence and these sorts of capabilities, they could weaponize it against us. Oh, carry on. They would feel extremely threatened by that, by a very concerted effort if it's trying to do that in a short amount of time, or if it's more plausibly on the horizon. This would cause them to do a similar type of project. And what would be the would this actually work? Well, you have information leakage issues, for instance. So if you're wanting to do that, you're going to need to convince those AI developers to go out to, you know, you do their last years of labor out in the middle of nowhere, Okay. It would need to just be, like, 5 Eyes countries people or people who can get security clearances. So it need to be people who are not easily extortable, for instance. So if they're Chinese nationals, they're probably more extortable because they often have family at home. So what are you doing with that talent? A lot of them want to be plausibly in the room where it happened. They don't want be left out, so they'll probably go back home to China. And then they'll work on the competing project there. Now, they're a substantial portion of the talent base. So you're I think you're shooting yourself in the foot if you're just saying, oh, it's only, you know, people born and raised in The US who can work on this sort of project. And they're all going to work in kind of unpleasant conditions. This wouldn't be secret. There's almost no way this would be secret. China would certainly would would very likely know. The such projects would be sabotageable as well. So it's sort of I I think there's some key if you're saying, well, we'll have it just be an industry or something like that, well, then you're not gonna have good information security. geable as well. So it's sort of I I think there's some key if you're saying, well, we'll have it just be an industry or something like that, well, then you're not gonna have good information security. You're gonna have to you're gonna have you're gonna have insider threat issues that people are extortable. You're going to have other classic computer security issues like they're using Slack. Slack is very easily hackable. They're using iPhones. IPhones are very easily hackable. So you can know what's going on there so you're not actually getting you're not actually having much in the way of secrets. So it it it sounds nice, but I think secrecy was very much an advantage for the Manhattan Project as well as having much more of the talent that can't go to other countries as easily. But I just don't think you have that. So there are ways in which AI is analogous to nuclear nuclear weapons and chemical weapons and biological weapons and some of these dual use technologies. But I don't think the Manhattan Project is is 1 of those things that's analogous. So sorry. So what this paper then is, well, so what is this sort of strategy? I think that the prospect of a superintelligence being imminent is extremely frightening to different actors. If it's imminent or if they have it, either way, or if it's being in the middle of being developed and it's arriving in a few months, that's extremely frightening if you miss out on that. So what do they want to do? They will either want to prevent such projects, or they will want to steal it. And so that looks like sabotage, for instance, for prevention. So how would they do that? Well, they may have some insider threats who could do some type of sabotage to sort of disrupt this type of project. They could do things like, say, snipe some of the power plants corresponding to the data center. Now your data centers don't work. They can do that from some miles away. Is was it China? Was it Russia? Was it a US citizen? You know, it's fairly unclear. There's a lot of ways they can have low attribute ability to prevent this sort of thing from happening. So this is a this is a I think that the fact that information you can't do a secret project really well. I think it's a is a substantial barrier. And then also the sabotageability is a substantial barrier, as well as how offensive and nuts you seem if you're saying, we're going to build superintelligence to and it's going to be explosive. Like, you're using superintelligence in, you know, a fixed sense. I think this would be destabilizing. China would reason, if The US controls it, then they could weaponize it against us, and we get crushed. Or they don't control it because they lose control of it in this process, in which case, we also want to prevent it. Either way, we want to prevent it, provided that they take this AI stuff seriously. And The US would reason the same about China. And Russia, which doesn't have a hope of competing, would definitely be wanting to prevent each other. And I think similarly for other nuclear states and other states that have substantial cyber capabilities. ia, which doesn't have a hope of competing, would definitely be wanting to prevent each other. And I think similarly for other nuclear states and other states that have substantial cyber capabilities. So this could lead to some type of deterrence dynamic where they make some attempt for getting superintelligence to get closer. But then other countries start to express very strong preferences against it. They say, if you do that, you know, we'll get very you get very mad. You know, there might be a skirmish or something like that. But then this may be something that pressures them to move more toward a verification regime, where they aren't make trying to make some bid for going having some sort of intelligence explosion, having AIs sort of do automated AI research, really quickly, like spinning up, you know, 100,000 AI instances to do AI research really quickly, and that could bring you from AGI to superintelligence in a short period of time. So I think that's a key dynamic. The fact the the extent to which it's it's destabilizing, I think that strategy needs to keep that in mind. So there may be cooperation, but it may be through coercion by saying, we're not gonna allow for this type of the this type of trajectory or you to you to make this bid for global dominance. And that that could give way to something more multilateral, and provide some strategic stability. So overall, with the paper, we we talk about 3 parts. In in the nuclear era, we had deterrence through mutual assured destruction. They don't use nukes because we can hit them back. They don't do the super in this case, this is kind of like preventing Iran's nuclear program in some way, that nobody's wanting each other to get, like, the nuclear bomb first or, like, a huge stockpile of nuclear bombs first. So there's preventing that from coming into existence. In the in the nuclear, we also had nonproliferation of fissile materials. We didn't want fissile materials being spread to to rogue actors, and we didn't want people having a poor man's atom bomb that would be very destabilizing and cause lots of catastrophes. And then we also had containment of the Soviet Union in the geopolitical competition between the 2. And I think for for for AI, we also have a deterrence thing. We also have nonproliferation, in this case, of of AI chips to rogue actors like North Korea or Iran or adversaries through export controls. And we also have competitiveness with China. Instead of it being containment of the Soviet Union, this would be competition with China. And how do we improve our competitiveness? Well, we wanna be you know, we want energy for AI data centers. We want secure supply chains so that if Taiwan is invaded, our AI chips aren't cut off. We want secure supply chains for robotics. Because if there is a US China conflict, then a lot of the a lot of that supply chain is currently in China, so they're very vulnerable. So those are some basic things to improve competitiveness. So it's making competitiveness not be, let's be the first to build super intelligence, which is what the sort of Manhattan project strategy pushes toward.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.