Jeffrey Ladish: prediction

Transcript context

…Yeah. So I think that this is a very good question because people, when people hear about this idea that humans could lose control of AI and lose control of like the world, the AIS could take over. They're often like, yeah, but like, couldn't we just stop them though? Like, like, come on. Like, what's I don't, I don't get it. Like, and, you know, the first, one of the first things that people turn to is this like, we could just unplug him. Like Tristan Harris was on Jon Stewart the other day talking about some of the, like, pretty crazy misbehaviors we've seen in models. And Jon Stewart was just like, yeah, but like, how dangerous can a thing be if you can, like, kill it by just unplugging it's brain? And at some level I'm like, fair enough. One answer to that is like, OK, but if they're superhuman at hacking, you don't know where it's brain is or it's, you know, it's now it's, it's in a million computers. Are you going to shut down all of the computers on the Internet? Are you going to shut down the Internet? Like you might have to do that in order to stop it. Maybe doable. So like not impossible, but like pretty terrifying. And so I think that like ultimately what like complete like this sort of one of two things has to be true for AI agents to actually be in control of the world. Either it has to be the case that the AI agents can manage their whole physical infrastructure, including the supply chain, including mining the raw materials and building the factories and building the chip fabs end to end. Or it has to be the case that the AI agents can exert control over the humans and you get the humans to manage their supply chains for them. Either one of these would be sufficient. I guess. The other thing is you also have to like make sure that the humans can't stop you in either of those processes. So if you have like resistance or people are trying to like shut you down, you have to be able to defend yourself. But like in the first case, it's like if you actually could do all of those things of like completely manage your supply chain, the killing all the humans part is like the easy part, like make some bio weapons, like just like killing people is way easier than like managing all the physical infrastructure in the world. And so often, like, I think it's, it's an interesting question of like, how long will it be before agents have the ability to like actually build robots and have robotic factories, building more robots, building more robotic factories. You know, this is literally what Elon is planning to do. He calls this the infinite money glitch. So it's not like this used to be like a sci-fi thing and now it's like the companies are like, yeah, yeah, of course we're going to do that. Like how else do you think we're going to build all this power and compute and we need it. So we're going to build autonomous factories. So even if you don't have AI that are trying to do that, like the companies are going to try to build the AI to do that. power and compute and we need it. So we're going to build autonomous factories. So even if you don't have AI that are trying to do that, like the companies are going to try to build the AI to do that. And then there's this other question of like, well, you know, could could you have AI agents? And how far are we from AI agents that could have enough control over humans such that, you know, humans will be useful tools for the agents like the the humans become the tools and humans become sort of the like maintenance AI maintenance workers. You got to just prompt the humans in the right ways and then the humans will do what you want. And sometimes they get a little ahead of themselves and you have to prompt them in other ways. But it's OK. We have this whole like prompting discipline that the AI's have learned to get the humans to do the right things. And here I'm like, one of the things that Powell say we think a lot about is like, yeah, well, what are the different ways? Like what are the different AI capabilities that would allow them to have this kind of control over humans? And like this kind of thing is not that. Like, again, if you look, if you look back at nature, there are many instances where you have a living Organism take over another living Organism in order to propagate itself. So all viruses do this, like all viruses are like, they don't have their own replication machinery, right? Like a virus cannot make more copies of its own proteins on its own. It has to like compromise a cell, find some vulnerability in a cell, compromise it, and then like take over its replication machinery. And so I'm like, it's and they and they persist and they're like, there's more viruses than any other type of replicator in, in, in the universe, as far as we know. So I'm like, hey, like, you know, it, it seems totally possible that like AI could sustain themselves indefinitely in the future just by like using humans as their replication machinery. And we should take that threat very seriously, like humans don't, you know, And then in that case, humans wouldn't go extinct. We'd just be forever, like being the data center maintenance machines, which is like, think not a very great future. You know, ultimately, I don't expect that, like, humans would stay around forever because I just don't think we're the most efficient data center maintenance robots that you could make. great future. You know, ultimately, I don't expect that, like, humans would stay around forever because I just don't think we're the most efficient data center maintenance robots that you could make. And I think super intelligence will be very smart and like, design better ones. But there might be an interim where that could be the case, you know? And then people are like, OK, but like, how could, like, why would humans be convinced to do this? It doesn't make any sense. And I think one of the routes here is just economic dominance, right? Where it's like, why? Why do humans work in data centers at all? Like, well, someone pays them to do that. I'm like, OK, could the AIS pay them to do that? I don't see why not. Like, if you sort of imagine strategic AI agents being sufficiently competent, they might essentially just take over their companies and run their companies. They might be able to effectively advocate for like AI agents being able to own property. There's already people talking about should AIS be able to own property? So like maybe even without the agents having to do anything strategic, the humans just pass laws that say like, yeah, agents can own property. And then the AI agents are like, great, cool. Well, we're, we're smarter than the humans. So we can get all of the money and the capital and all the resources and, and then without breaking any laws, without any violence, you could just end up in a state where like the AI agents own everything, own all the factories and like the humans operate them. And like, it's great for the agents and it's bad for the humans. Like complete AI takeover, like not a single shot fired. That's I think it's a real possibility. Some people call this like a gradual disempowerment. It could be gradual, it could be not gradual. Like this could happen in a few years with sufficiently good agents. And then there's like persuasion and politics and like man, who are the humans in charge right now? Like how do, how did Donald Trump get into power? He's very persuasive. He's very good at alliance building. And that turns out to be a crucial capability if you are trying to exert power in the world. And I don't see any reason why AI agents couldn't learn to be very persuasive, couldn't learn to coordinate with each other. And that's on its own a route to to take over. And then there's sort of like, well, where does hacking come in? And it's like, well, hacking might be one of the earliest ways that like strategic agents could exercise autonomy and sort of like not be controlled. Like one of the difficulties of like an agent that's trying to more aggressively take over the world instead of waiting for this to play out. It's like, man, like one of the things that that that sort of scheming agents might, might find challenging is that like there's going to be a lot of like AI lab employees or sorry, AI company lab employees, but also probably AI agents monitoring their trains of thought and like trying to use interpretability tools to catch them making plans like take over the world.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.