Dwarkesh Patel: observation

Transcript context

…Alignment during the intelligence explosion is not a years-long bureaucratic process. It's more like a war, with a fog of war. Is it safe to do the next OOM? We're three OOMs into the intelligence explosion, and we don't fully understand what's happening. Our generalization-scaling curves don’t look great, some automated AI researchers say it's fine, but we don't quite trust them. AIs might start doing problematic things, but we hammer it out, and then it's fine. Should we go ahead? Should we take another six months? Meanwhile, China might steal the weights or deploy a robot army. It's a crazy situation, relying more on a sane chain of command than a deliberative regulatory scheme. Although I wish we could do that more deliberative regulatory scheme. This is the thing with private companies too. Private companies claim they'll do safety, but it’s rough in a commercial race, especially for startups. Startups are startups. They aren't fit to handle WMDs. I’m coming closer to your position but… Let’s talk about the responsible scaling policies. I was told by people advancing this idea — because they know I’m a libertarian-type person and the way they approached me was like this — that the way to think about it was that it's fundamentally a way to protect market-based development of AGI. If you didn’t have this, there would be misuse and lead to nationalization. The RSPs are a way to ensure a market-based order with safeguards to prevent things from going off the rails. It seems like your story is self-consistent. I know this was never your position, so I’m not looping you into this. But it's almost like a motte-and-bailey argument. Here’s what I think about RSP-type stuff and current safety regulations. They’re important for helping us figure out what world we’re in and flashing the warning signs when we’re close. The story we’ve been telling is what I think the modal version of this decade is. There are many ways it could be wrong. We should talk about the data wall more. There’s a world where this stuff stagnates or we don’t have AGI. The RSPs preserve optionality. Let’s see how things go, but we need to be prepared if the red lights start flashing. If we get the automated AI researcher, then it’s crunch time.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.