Dario Amodei: evaluation

Transcript context

…Let me now ask you about making AI go well. It seems like whatever vision we have about how AI goes well has to be compatible with two things: 1) the ability to build and run AIs is diffusing extremely rapidly and 2) the population of AIs, the amount we have and their intelligence, will also increase very rapidly. That means that lots of people will be able to build huge populations of misaligned AIs, or AIs which are just companies which are trying to increase their footprint or have weird psyches like Sydney Bing, but now they’re superhuman. What is a vision for a world in which we have an equilibrium that is compatible with lots of different AIs, some of which are misaligned, running around? I think in “The Adolescence of Technology”, I was skeptical of the balance of power. But the thing I was specifically skeptical of is you have three or four of these companies all building models that are derived from the same thing, that they would check each other. Or even that any number of them would check each other. We might live in an offense-dominant world where one person or one AI model is smart enough to do something that causes damage for everything else. In the short run, we have a limited number of players now. So we can start within the limited number of players. We need to put in place the safeguards. We need to make sure everyone does the right alignment work. We need to make sure everyone has bioclassifiers. Those are the immediate things we need to do. I agree that that doesn’t solve the problem in the long run, particularly if the ability of AI models to make other AI models proliferates, then the whole thing can become harder to solve. I think in the long run we need some architecture of governance. We need some architecture of governance that preserves human freedom, but also allows us to govern a very large number of human systems, AI systems, hybrid human-AI companies or economic units. So we’re gonna need to think about: how do we protect the world against bioterrorism? How do we protect the world against mirror life? Probably we’re gonna need some kind of AI monitoring system that monitors for all of these things. But then we need to build this in a way that preserves civil liberties and our constitutional rights. So I think just as anything else, it’s a new security landscape with a new set of tools and a new set of vulnerabilities. My worry is, if we had 100 years for this to happen all very slowly, we’d get used to it. We’ve gotten used to the presence of explosives in society or the presence of various new weapons or the presence of video cameras. We would get used to it over 100 years and we’d develop governance mechanisms. We’d make our mistakes. My worry is just that this is happening all so fast. So maybe we need to do our thinking faster about how to make these governance mechanisms work. It seems like in an offense-dominant world, over the course of the next century—the idea is that AI is making the progress that would happen over the next century happen in some period of five to ten years—we would still need the same mechanisms, or balance of power would be similarly intractable, even if humans were the only game in town. I guess we have the advice of AI. But it fundamentally doesn’t seem like a totally different ball game here. If checks and balances were going to work, they would work with humans as well. If they aren’t going to work, they wouldn’t work with AIs as well. So maybe this just dooms human checks and balances as well.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.