Dwarkesh Patel: belief

Transcript context

…Yeah. And so, for example, I’m not a lawyer, I don’t know the details about how this would work out, but I have a sort of high-level strategic picture of the fight between the White House and the CEO. And the strategic picture is basically the White House can sort of threaten, “here’s all these orders I could make, Defense Production Act, blah, blah, blah. I could do all this terrible stuff to you and basically disempower you and take control”. And then the CEO can threaten back and be like, “here’s how we would fight it in the courts, here’s how we would fight it in the public. Here’s all this stuff we would do”. And after then they both do their posturing with all their threats, then they’re like, “okay, how about we have a contract that instead of executing on all of our threats and having all these crazy fights in public, we’ll just come to a deal and then have a military contract that sets out who gets to call what shots in the company”. And so that’s what we depict happening is that they don’t blow up into this huge power struggle publicly, instead they negotiate and come to some sort of deal where they basically share power. And there is this oversight committee that has some members appointed by the President and also the CEO and his people. And that committee votes on high level questions like “what goals should we put into the superintelligences?”. So, we were just getting lunch with a prominent Washington, D.C. political journalist, and he was making the point that when he talks to these congresspeople, when he talks to political leaders, none of them are at all awake to the possibility even of stronger AI systems, let alone AGI, let alone superhuman intelligence. I think a lot of your forecast relies on, at some point, not only the US President, but also Xi Jinping, waking up to the possibility of a super intelligence and the stakes involved there. Why think that even when you show Trump the remote worker demo, he’s going to be like, “oh, and therefore in 2028, there will be a super intelligence. Whoever controls that will be God emperor forever”. Maybe not that extreme, but you see what I’m saying. Why wouldn’t he just be like, “there’ll be a stronger remote worker in 2029, a better remote worker in 2031”? Well, to be clear, we are uncertain about this, but in our story, we depict this sort of intense wake up happening over the course of 2027, mostly concurrently with the AI companies automating all of their R&D internally and having these fully autonomous agents that are amazing autonomous hackers and stuff like that, but then also actually doing all the research. And part of why we think this wakeup happens is because the company deliberately decides to wake up the president. You could imagine running the scenario with that not happening. You can imagine the companies trying to sort of keep the president in the dark. I do think that they could do that. I think that if they didn’t want the President to wake up to what’s going on, they might be able to achieve that. Strategically though, that would be quite risky for them. Because if they keep the President in the dark about the fact that they’re building superintelligence and that they’re actually completely automated their R&D and it’s getting superhuman across the board, and then if the President finds out anyway somehow, perhaps because of a whistleblower, he might be very upset at them and he might crack down really hard and just actually execute on all the threats and nationalize them and blah, blah, blah. They want him on their side. And to get him on their side, they have to make sure he’s not surprised by any of these crazy developments. And also, if they do get him on their side, they might be able to actually go faster. They might be able to get a lot of red tape waived and stuff like that. And so we made the guess that early in 2027, the company would basically be like, ‘We are going to deliberately wake up the president and scare the president with all of these demos of crazy stuff that could happen, and then use that to lobby the President to help us go faster and to cut red tape and to maybe slow down our competitors a little bit and so forth.’…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.