Robert Wright: belief

Transcript context

…... with like, what's your positive vision for the future? If you, you know, if you could navigate all these pitfalls, where do you think we want to end up? Okay. Before I cheer you up, let me just... Let's, let's dwell in the depths of despair just a little more to, just to flesh out what you were... That, that quote, like, we can do this the easy way or the hard way. So there's a chapter on, m-to a certain extent on Nick Bostrom's thinking called The Singularity in the Singleton. Singleton is this idea of a global coordinating mechanism, global governance. It, it could be anything. You know, it could be, you know, a de- global democracy with very decentralized power, which would be my preference, run by humans. It could be run by AIs. It could be a totalitarian nightmare. It could be anything. But, but a glo- a mechanism, some mechanism of global coordination, he calls it, or, or global governance. And I had him on my podcast, and we're, we're pretty much on the same page about this. You know, we, we, we agree that in recognition of the non-zero-sum dynamics that, you know, among nations and so on that are posed by technologies, especially AI, a certain kind of global governance makes sense. But as, as he fleshes out in his, his, his famous book, Superintelligence, you can imagine some dark paths to global coordination of a dark kind, right? Like, uh, you know, AI is used to stage some coup and just seize power. The AI seizes power and so on. My view is that one or another of these scenarios is indeed very likely, and that's what I mean when I say we can do this the easy way or the hard work. In other w- or the hard way. In other words, humans can consciously steer us toward a system of international governance of this technology that involves as little centralization as pow- of power as possible, but as much as is necessary to keep the technology under control. And even in the event that it starts playing a very large, large, large role in governance itself, still maintaining a kind of a win-win relationship with it, right? So that's what I meant by the easy way or the hard way. Now, what are the chances that we get there the easy way? As I said, I think a certain amount of international conflict is gonna have to subside, and that may take something you could call moral progress. I talk a lot about cognitive empathy in the book. I don't mean emotional empathy. I don't mean like feeling their pain. You know, I just mean having a clear enough understanding of the perspective of other people and groups, including adversaries, that you can r- you can intelligently play non-zero-sum games with them and work things out that are in your own self-interest and, by virtue of being non-zero-sum, are of mutual benefit. So I, I do think we're gonna have to make an advance of that. ro-sum games with them and work things out that are in your own self-interest and, by virtue of being non-zero-sum, are of mutual benefit. So I, I do think we're gonna have to make an advance of that. The cognitive biases that impede cognitive empathy are now pretty well known. I talk about them, attribution error and so on. And so we're gonna have to have progress. As I said, I think, I think that can be part and parcel of just looking out for our own cognitive health as AI plays a more pervasive role in our lives. I like the term cognitive sovereignty. I didn't come up with it, but I think being mindful of how persuasive and subtly influential AI could be and, and perhaps not with your welfare quote in mind, I, I, I think it's good for us to, to tend to our kind of, uh, psychological robustness, and I d- I do think cultivating cognitive empathy can be part of that. I also think, you know, Ilya Sutskever says something, he said something in a TED Talk that I quote to the effect of like, you know, what he thinks is that at some point an awareness of the magnitude of this will dawn and there will be new, a new inclination toward collaboration with other human beings in recognition that increasingly we humans are put in a non-zero-sum situation by the AI, right? Like, it, it's in our interest to coordinate. And doesn't mean we can't have a non-zero-sum relationship with the AI at all. It doesn't mean it has to be permanently in, in the... It, it, the, it's, it's not necessarily an enemy, but it has the potential to be bad for us in ways that should, I think, exert a cohesive effect on us. And I think he's right. It, it's... I mean, first of all, let me say, there's been, you know, very recently, just Mythos alone shows you what a big difference one innovation can make, okay? Two things that you might not have predicted six weeks ago. One is there, there is now a, a, there is an official dialogue between the US and China on AI safety. I think that's a consequence of Mythos. I think the focus of it is more narrow for now than I'd like, but it's happening. And then secondly, uh, the Trump administration, after Trump had ridiculed a, a Biden administration executive order that I think is similar in spirit to this new one that Trump has issued. You know, the Trump, the Trump administration has issued an executive order about the government vetting particularly powerful models. Now, I think that kind of arrangement is, is something to keep an eye on. You know, whenever the government gets close to the power of AI, you know, that's one of the things you wanna be careful about. Still, I personally think some of this is gonna have to happen, and my main point is none of it... none of this was on, was happening two months ago, just two months ago, and it didn't even take a catastrophe, right? It just took a... I'm not even sure Mythos qualifies as a near miss, but it, it, it got our attention to some of the nefarious potential of this stuff, and it got us to focus. And I think, you know, AI is so different from technological challenges of the past.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.