Robert Wright: prediction

Transcript context

…... that's like, oh yeah, good point. You know, we don't have a lot of, we don't have a lot of ways to control. Yeah, so I, that sounds like, uh, you know, we're, now we're talking about a different level of evolution, like selection among competing models in the marketplace. Let me quickly, before I elaborate, quickly say one thing. As long as you mentioned Dwarkesh, you know, the much discussed Richard Sutton episode, I, I personally think the Rosetta Stone to that, in terms of understanding where Sutton was coming from, is that, I don't know this to be true, but I strongly suspect it based on the conversation, is that he, he is a hardcore Skinnerian, you know, B.F. Skinner behaviorist, and thinks of the mind as a blank slate. So he thinks, I th- I would guess, that training is all learning, right? That, that, that i- it really, intelligence is general in the broadest sense, you know? And you don't need much in, you know, pre-built equipment built by natural selection. But anyway, that, that's an aside. I'm not sure of that interpretation, but I, uh, it kinda sounded that way to me. Yeah, so as for alignment, I mean, I think alignment research is, you know, very much worth pursuing. There's a couple of reasons I wouldn't wanna put all my eggs in that basket, and one of them is that, you know, other forces are gonna have a voice i- in what kinds of models we actually wind up with. Leaving aside the question of whether the perfectly aligned model is, in principle, possible and we know how to make it, and I'm not, I'm not confident that that will ever happen anyway. But it's like when, when you think about the assumption that that will save the day, it seems to me the assumption is that there will be a pretty s- you know, centralized form of control, right? Uh, or that there will be one model that rules them all, and maybe there will be. Uh, certainly there needs to be some kind of control, and certainly one of the great dilemmas with AI is, of course, the fact that in some ways you want, you know, authority, governance. You can point to reasons you want that. At the same time, it's a very powerful technology that you wouldn't want to see super centralized control over because, you know, the, the person or people controlling it might abuse it. But in any event, yeah, let's talk a little about that second level of evolution, selection among models. Now, in, in certain ways, I think if you ask what does it encourage, uh, the selective process where, where, you know, consumers say, "I like this model," and businesses say, "I like that model," and then maybe powerful actors have a say, especially if there's market concentration or government control. But I think, you know, the truth is that the market doesn't want an aligned model in the strictest sense of the term. tors have a say, especially if there's market concentration or government control. But I think, you know, the truth is that the market doesn't want an aligned model in the strictest sense of the term. So, you know, we have seen that these things have, as prescient people predicted, apparently some kind of power-seeking tendencies, uh, the, the, the capacity for strategic deception, and when you think about it, even if they, they didn't have that, quote, "naturally," in other words, even if, you know, just as a, as a kind of form of intelligence, they didn't, they didn't come to p-pursue subordinate goals such as power seeking or deception, I think there would still be a market for those things, right? I mean, like, on the deception front, um, you know, i-if they're gonna be our agents out in the world, like representing us on social media, maybe in negotiations, well, we don't want a, a perfectly accurate representation of us, right? Like, who wants that? Who wants the actual Bob to be seen on social media? You want, at best, a selective representation, and if some, and if some agent is negotiating for me, I don't want the agent to say, "You know, I'll, I'll level with you. You know, Bob doesn't have any options other than you. Nobody's made him an offer other than you," right?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.