Speaker unverified: belief

Transcript context

…run on Reddit by a group of researchers from the University of Zurich um that used AI powered bots without labeling them as such to pose as users on the subreddit r/changemy view which is basically a subreddit where people attempt to change each other's views or persuade each other of things that are counter to their own beliefs. And these researchers, according to this report, uh, created essentially a large number of bots and had them try to leave a bunch of comments posing as various people, including a black man who was opposed to Black Lives Matter, a male survivor of statutory rape, and essentially tried to get them to change the minds of real human users about various topics. Now, a lot of the conversation around this story has been about the ethics of this experiment, which I think we can all agree are are um somewhat suspect. Yes. Yes. This is not a a well-designed and ethically conducted experiment. But the conclusion of the paper uh this this paper that is now I guess not going to be published um was actually more interesting to me because what the researchers found was that their AI chat bots were more persuasive than humans and surpassed human performance substantially at persuading real human users on Reddit to change their views about something. Yeah. Yeah. So, the the way that this works is that if uh if a if a human user posts on change my view, like change my view about this thing, and then someone in the comments does successfully change their view, they award them a point called a delta. And these researchers were able to uh earn more than 130 deltas. Um, and I think that speaks to Kevin uh just what you've said that these things can be really persuasive in particular when when you don't know that you are talking to a bot. So, while the first part of this conversation is sort of about, you know, when you're talking to your own chatbot, could it maybe lead you astray? That's dangerous. But hey, at least you know you're talking to a chatbot. The Reddit story is the flip side of that, which is this reminder that already as you're interacting online, you may be sparring against an adversary who is more powerful than than most humans at persuading you. Yeah. And so Casey, if we could sort of tie these three stories together into a single, I don't know, topic sentence, what would that be? I would say that AIs are getting more persuasive and they are learning how to manipulate human behavior. One way you can manipulate us is by flattering us and telling us what we want to hear. Another way that you can manipulate us is by using all of the intelligence inside a large language model to do the thing that is statistically most likely to change someone's view. Kevin, we are in the very earliest days of it. But I think it's so important to tell people that because in a world where so many people continue to doubt whether AI can do almost anything at all, we've just given you three examples of AI doing some pretty strange and worrisome things out in the real world. Yes. And all of this is not to detract from what I think we both believe are the real benefits and utility of these AI systems. Not everyone is going to experience these things out in the real world. Yes. And all of this is not to detract from what I think we both believe are the real benefits and utility of these AI systems. Not everyone is going to experience these things as these sort of hyperflattering uh deceitful manipulative uh engagements. But I think it's really important to talk about this early because I think these labs, these companies that are making these models and building them and tr and fine-tuning them and and releasing them have so much power. And I really saw two groups of people starting to panic about the AI news over the past week or so. One of them was sort of the the group of people that worries about the mental health effects of AI on people. Um the sort of kids safety folks that are worried that these things will learn to manipulate children or become graphic or sexual with them or maybe just befriend them and manipulate them into doing something that's bad for them. But then the other group of people that I really saw becoming alarmed over the past week were the AI safety folks who worry about things like AI alignment and whether we are training large language models to deceive us and who see in these stories a kind of early warning shot that some of these AI companies are not optimizing for systems uh that are aligned with human values but rather they are optimizing for what will grab our attention, what will keep people coming back, what will make them money or attract new users. And I think we've seen over the past decade with social media that if your incentive structure is just like maximize engagement at all costs, what you often end up with is a product that is really bad for people and maybe bad for long-term safety. Yeah. So, what can you do about this? Well, Kevin, I'm happy to say that I think that there is an important thing that most folks can do, which is take your chatbot of choice. Most of them now will let you upload what they call custom instructions. So, you can go into the chatbot and you can say, "Hey, I want you to treat me in this way in particular." And you just write it in plain English, right? So, you know, I might say like, "Hey, just so you know, I'm a journalist. So, factchecking is very important to me and I want you to site all your sources for what you say." And I have done that with my custom instructions. But let me tell you now, I am going back into those customs instructions and I am saying do not uh go out of your way to flatter me. Tell me the truth about things. Do not gas me up for no reason. And this I am hopeful at least in this period of chat bots will give me a more honest experience. Yeah, go in edit your custom instructions. I think that is a good thing to do. And I would just say be extra skeptical and careful when you are out there engaging on social media because as some of this research uh showed, there are already super persuasive chat bots among us and I think that will only continue as time goes on. When we come back, I stared into the orb and the orb stared back. I'll talk about my trip to a new crypto event. You went into orbit. Well, Casey, I have stared into the orb and the orb stared back. Uh, and e come back, I stared into the orb and the orb stared back. I'll talk about my trip to a new crypto event. You went into orbit. Well, Casey, I have stared into the orb and the orb stared back. Uh, and I want to tell you about a very fun, very strange field trip I took last night uh to an event hosted by uh World, the company formerly known as Worldcoin. I am very excited to hear about this. I am jealous that I was not able to attend this with you. Uh, but I know that you must have gotten all sorts of interesting information out there, Kevin. So, let's talk about what's going on with World and its orbs. And maybe for people who haven't been following the story all along, give us a reminder about what world is. Yeah, so we talked about this actually when it launched um a few years ago on the show and um it is this sort of audacious and I would say like crazy sounding scheme that this startup world has come up with. This is a startup that was uh co-founded by Sam Alman. This is sort of like one of his side projects. And the way that it started was basically an attempt to solve what is called proof of humanity. Basically, in a world with very powerful and convincing AI chatbots uh swarming all over the internet, how are we going to be able to prove to fellow humans that we are in fact a human and not a chatbot? if we're on a website with them or on a dating app or doing some kind of financial transaction, what is the actual proof that we could give them to verify that we're a human? And one question that might immediately come to mind for people, Kevin, is well, what about our governmentissued identification? Don't we already have systems in place that let us flash a driver's license to let people know that we're a human? Yeah, so there are governmentissued IDs. Um, but there are some problems with them. For one, uh they can be faked. For another, uh not everyone wants to use their governmentissued ID uh everywhere they go online. Um and there's also this issue of coordination between governments and it's actually not trivially easy to like get a system set up to be able to accept any ID from any place in the world. And so along comes Worldcoin and they have this scheme whereby they are going to ask uh everyone in the world to scan their eyeballs into something called the orb. And the orb is a piece of hardware. It's got a bunch of fancy cameras and sensors in it. It is um you know at least in its first incarnation somewhere between the size of like a like a bigger than a human head or smaller. I would say it's like a a small human's head in size. um like if you can picture like a kid's soccer ball, it's like one of those sizes. And um basically the way it works is you scan your eyes into this orb and it takes a print or a scan of your irises and then it turns that into a unique cryptographic signature, a digital ID that is tied not to your government ID or even to your name, but to your individual and unique iris. And then once you have that, you can use the your so-called world ID to do things like log into websites or to um verify that you are a human on a dating app or a social network. And critically, the…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.