Robert Wright: belief

Transcript context

…tors have a say, especially if there's market concentration or government control. But I think, you know, the truth is that the market doesn't want an aligned model in the strictest sense of the term. So, you know, we have seen that these things have, as prescient people predicted, apparently some kind of power-seeking tendencies, uh, the, the, the capacity for strategic deception, and when you think about it, even if they, they didn't have that, quote, "naturally," in other words, even if, you know, just as a, as a kind of form of intelligence, they didn't, they didn't come to p-pursue subordinate goals such as power seeking or deception, I think there would still be a market for those things, right? I mean, like, on the deception front, um, you know, i-if they're gonna be our agents out in the world, like representing us on social media, maybe in negotiations, well, we don't want a, a perfectly accurate representation of us, right? Like, who wants that? Who wants the actual Bob to be seen on social media? You want, at best, a selective representation, and if some, and if some agent is negotiating for me, I don't want the agent to say, "You know, I'll, I'll level with you. You know, Bob doesn't have any options other than you. Nobody's made him an offer other than you," right? nt is negotiating for me, I don't want the agent to say, "You know, I'll, I'll level with you. You know, Bob doesn't have any options other than you. Nobody's made him an offer other than you," right? Like, a-and, and if they a-- and if they ask has anybody made him an offer, you want an agent that will not disclose the truth, right? So th-there's a lot of cases of that, I think, and, you know, for that matter, what do we want in a friend, right? We don't want a friend who's always leveling with us, right? We, we don't... You know, uh, we, we-- A good friend is selective in their candor and is, is healthy in the feedback they give, but we, we don't wanna hear the brutal truth about how we look and so on every single day. You know, I could tell a similar story about power seeking. In, in a certain sense, if you turn an agent loose on social media with broad instructions like, "Just use it to maximize our revenue," you'll realize that what you really want is for it to, to be good at sensing power, what other people on social media are powerful, currying favor with them and, and blah, blah, blah, and amassing power. So even if the machines didn't naturally do that, we, we'd want these things, and I think that complicates, uh, the, the, the task of aligning. But, but, but moreover, I think it should just alert us in, I hope, a constructive way to what a powerful role we are playing here as individual consumers, for example. You know, I, I worry about the tribalizing tendencies of AI, which to some extent are a byproduct of the sycophantic tendencies, right? To say, "Hey, you're great," interesting point, is the same kind of, you know, reinforcement, uh, you give a person, uh, when you say, "Hey, by the way, you're right about this and the other person's wrong. You're right. Your spouse is wrong in this conflict. Your nation's right. The other nation's wrong." The world doesn't need more of that, uh, and yet companies that want to optimize for engagement, and what company doesn't wanna do that? I mean, if you make candy bars, you want people to spend a lot of time eating your candy bars. Companies that do that, um, are gonna be giving us tribalizing AIs unless they're careful not to, and I think to some extent the ball is in our court. We need to be aware that, you know, AIs could have an unfortunate effect on our psychology, on our self-conception, uh, unfortunate for the world, uh, and I think especially unfortunate now that I think, you know, this technology needs to be governed by a true global community. We need to start approaching the whole thing as a, as a planet. I guess I'm, I'm happy to say that I think, you know, to a large extent, selecting AIs that are good for the world can be selecting AIs that are good for you. You know? The same reason, like, people meditate, people, people do mindfulness meditation 'cause it calms, it calms them down. They do fewer ill-advised things. Their life is, on balance, better. Well, that's also good for the world 'cause you're, you're, you're, you're creating less needless an-antagonism. t calms them down. They do fewer ill-advised things. Their life is, on balance, better. Well, that's also good for the world 'cause you're, you're, you're, you're creating less needless an-antagonism. So it can happen that self-help is good for the world, and I th- I'm hoping that as we exert selective pressure on AI models, we will, I mean, be, be more conscious maybe than typically of the effect on the world because I think we're, we're approaching a crossroads where, where the world really can't afford, afford to continue to be so divided, and neither can individual nations. But I'm also hoping that if we even make wise decisions from the point of view of our own psychological well-being, that will have, uh, good effects in the broader communities.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.