Dan Hendrycks: evaluation

Transcript context

…geable as well. So it's sort of I I think there's some key if you're saying, well, we'll have it just be an industry or something like that, well, then you're not gonna have good information security. You're gonna have to you're gonna have you're gonna have insider threat issues that people are extortable. You're going to have other classic computer security issues like they're using Slack. Slack is very easily hackable. They're using iPhones. IPhones are very easily hackable. So you can know what's going on there so you're not actually getting you're not actually having much in the way of secrets. So it it it sounds nice, but I think secrecy was very much an advantage for the Manhattan Project as well as having much more of the talent that can't go to other countries as easily. But I just don't think you have that. So there are ways in which AI is analogous to nuclear nuclear weapons and chemical weapons and biological weapons and some of these dual use technologies. But I don't think the Manhattan Project is is 1 of those things that's analogous. So sorry. So what this paper then is, well, so what is this sort of strategy? I think that the prospect of a superintelligence being imminent is extremely frightening to different actors. If it's imminent or if they have it, either way, or if it's being in the middle of being developed and it's arriving in a few months, that's extremely frightening if you miss out on that. So what do they want to do? They will either want to prevent such projects, or they will want to steal it. And so that looks like sabotage, for instance, for prevention. So how would they do that? Well, they may have some insider threats who could do some type of sabotage to sort of disrupt this type of project. They could do things like, say, snipe some of the power plants corresponding to the data center. Now your data centers don't work. They can do that from some miles away. Is was it China? Was it Russia? Was it a US citizen? You know, it's fairly unclear. There's a lot of ways they can have low attribute ability to prevent this sort of thing from happening. So this is a this is a I think that the fact that information you can't do a secret project really well. I think it's a is a substantial barrier. And then also the sabotageability is a substantial barrier, as well as how offensive and nuts you seem if you're saying, we're going to build superintelligence to and it's going to be explosive. Like, you're using superintelligence in, you know, a fixed sense. I think this would be destabilizing. China would reason, if The US controls it, then they could weaponize it against us, and we get crushed. Or they don't control it because they lose control of it in this process, in which case, we also want to prevent it. Either way, we want to prevent it, provided that they take this AI stuff seriously. And The US would reason the same about China. And Russia, which doesn't have a hope of competing, would definitely be wanting to prevent each other. And I think similarly for other nuclear states and other states that have substantial cyber capabilities. ia, which doesn't have a hope of competing, would definitely be wanting to prevent each other. And I think similarly for other nuclear states and other states that have substantial cyber capabilities. So this could lead to some type of deterrence dynamic where they make some attempt for getting superintelligence to get closer. But then other countries start to express very strong preferences against it. They say, if you do that, you know, we'll get very you get very mad. You know, there might be a skirmish or something like that. But then this may be something that pressures them to move more toward a verification regime, where they aren't make trying to make some bid for going having some sort of intelligence explosion, having AIs sort of do automated AI research, really quickly, like spinning up, you know, 100,000 AI instances to do AI research really quickly, and that could bring you from AGI to superintelligence in a short period of time. So I think that's a key dynamic. The fact the the extent to which it's it's destabilizing, I think that strategy needs to keep that in mind. So there may be cooperation, but it may be through coercion by saying, we're not gonna allow for this type of the this type of trajectory or you to you to make this bid for global dominance. And that that could give way to something more multilateral, and provide some strategic stability. So overall, with the paper, we we talk about 3 parts. In in the nuclear era, we had deterrence through mutual assured destruction. They don't use nukes because we can hit them back. They don't do the super in this case, this is kind of like preventing Iran's nuclear program in some way, that nobody's wanting each other to get, like, the nuclear bomb first or, like, a huge stockpile of nuclear bombs first. So there's preventing that from coming into existence. In the in the nuclear, we also had nonproliferation of fissile materials. We didn't want fissile materials being spread to to rogue actors, and we didn't want people having a poor man's atom bomb that would be very destabilizing and cause lots of catastrophes. And then we also had containment of the Soviet Union in the geopolitical competition between the 2. And I think for for for AI, we also have a deterrence thing. We also have nonproliferation, in this case, of of AI chips to rogue actors like North Korea or Iran or adversaries through export controls. And we also have competitiveness with China. Instead of it being containment of the Soviet Union, this would be competition with China. And how do we improve our competitiveness? Well, we wanna be you know, we want energy for AI data centers. We want secure supply chains so that if Taiwan is invaded, our AI chips aren't cut off. We want secure supply chains for robotics. Because if there is a US China conflict, then a lot of the a lot of that supply chain is currently in China, so they're very vulnerable. So those are some basic things to improve competitiveness. So it's making competitiveness not be, let's be the first to build super intelligence, which is what the sort of Manhattan project strategy pushes toward. some basic things to improve competitiveness. So it's making competitiveness not be, let's be the first to build super intelligence, which is what the sort of Manhattan project strategy pushes toward. But instead, its competition is more, you know, market share in the in across the globe of people using your AIs as opposed to Chinese AIs in your supply chain security instead. So that's kind of what's in the paper at a high level. There's lots of other specific things in there, like assuming high levels of automation, what are ways that you distribute power, what are things about AI rights, what are reasonable alignment targets that are actually implementable compared to, you know, vague philosophical, you know, words like dignity or something like that. So we'll we'll touch on a lot of those in the the expert version of the superintelligence strategy. But hopefully, that gives gives some sense of its its content. So it's trying to for all of the key questions, we'll try and have some answer to what to do about AI and what to do about superintelligence.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.