Robert Wright: belief

Transcript context

…... manage that handoff well is gonna be extremely difficult. And so whatever we can do to maximize our chance, our being humanity broadly, but also Anthropic specifically, of doing that effectively, we should do. And if that means racing out to have as much lead as we possibly can, then that'll be good because that will give us this buffer that then we can burn down that lead that we have over other AI developers in that critical moment when we have that most powerful AI, so that hopefully us and our best AIs available at the time will sort of be the watchmaker that sets things on the right path, and then we can all live happily ever after. But we need as much time in that critical period as we can get, and so that's why we race now, so we can, you know, have as much time to be as thoughtful and, you know, cautious as we can be a- at that later critical phase. All of which mi- b- by the way, I think they're actually pretty clear-eyed. It's, like, maybe gonna be three to six months, you know, in 2028 or 2029. I think that's, like, re- literally how they're thinking about it. So I don't like those odds. I don't like that plan that much. I don't like the, our odds if, if that is the plan that we're gonna pursue. But I, I think that's a, that's, I think, a reasonably good... I mean, somebody, you know, uh, correct me if I'm wrong. I don't think too many Anthropic people have time for two-hour podcasts these days, but I'd love to be, you know, corrected if that's not the way they're thinking about it. But that, that's my sense right now as to the prevailing Anthropic thought. Yeah, I'd say a couple things. I mean, first of all, as was pointed out in this superintelligence strategy paper co-authored by Dan Hendricks and Eric Schmidt and Alexander... Is it Wang or... Do you pronounce, pronounce it Wang? The, the, the guy who started Scale AI, I think. Anyway, I, I think this was not emphasized enough in the paper, frankly. But w- one thing they said is, you know, if you're racing towards superintelligence, and there's these two superpowers, there comes a time when one of them is strongly incentivized to derail the leader's AI effort, and that may include things like bombing data centers or aggressive cyber acts. So it, there, there's an inherent kind of destabilization in approaching the superintelligence threshold if they both, if both parties buy the idea that that is gonna confer utter military hegemony. And it is the official position of Dario Amodei that, that, that it's gonna confer that and that we should Race headlong toward it, as I understand his position. So first of all, you're kind of inviting China to attack you and, or do something highly destabilizing. The second thing I'd say is, you know, some of the assumptions about alignment working out that we talked about at the beginning that are maybe not airtight, I think enter into the scenario you described. Not to mention, well, there's a lot of assumptions, a lot of assumptions that I think have to be right for that to be worked out. But the other thing I'd say is it just seems to me that once you appreciate the delicacy of the challenge, I mean, the, you gotta thread one kind of needle or another. That's a good comeback to my argument. If I say, "You guys are talking about threading a needle," they'll say, "Well, so are you. You're talking about, like, what? Rapprochment with China," blah, blah, blah. We, we gotta thread, we, you know, we, we have to overcome some serious seemingly formidable challenge in any scenario. But it does seem to me that once you appreciate that, maybe you should appreciate the logic behind just slowing down rather than speeding up. Like, as, you know, like maybe the more time we have to think about it, the better. The more time we have to develop the kind of international collaboration that would allow us to do this the easy way and so on. So that's the kind of reply I'd give to that. I mean, I will say even Anthropic in this latest paper about recursive self-improvement says at the end, m- I mean, it doesn't say what the headline said it said. The headline said it was calling for a global pause, which it wasn't, but it, they did say, you know, maybe the time will come to slow down or pause, and so we think we should now enter a period of studying what that would take. And my question asked, tell me if I'm being too big a jerk here, Nathan, but by the way, I hope you've, if, if I'd been a j-jerk up until now, you would've jumped in and said, "Bob, you're being a jerk," so. question asked, tell me if I'm being too big a jerk here, Nathan, but by the way, I hope you've, if, if I'd been a j-jerk up until now, you would've jumped in and said, "Bob, you're being a jerk," so. But the i-i- is, you know, my view is like, wait, you knew we're, you've been saying we, we'd get to this point of recursive self-improvement. Surely this isn't the first time it's occurred to you that things might moving s- might move so fast at that point that we should slow down or pause in advance. And you mean to tell me you haven't yet given any systematic thought to how we'd work that out? I mean, they're, they're just gonna kinda start the study process now? I don't, I don't, I don't know. I, I, I, I, I'd be curious to know what in, what dynamics led to that paper. I, I, I, I'd like to think there's pressure within Anthropic in favor of talking about a pause and a slowdown. Maybe Dario's not so enthusiastic because it's inconsistent with his, his advocated China policy. But in any event, I agree it's time that a lot of think tanks started focusing on this question of how, how do we, how do we coordinate globally, you know? To do things generally with AI, including slow down.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.