Speaker unverified: belief

Transcript context

…w that lots of people are using chat bots for this reason. Um but most companies have put in guard rails to prevent minors from doing this sort of thing, right? It turns out that Meta had not been and that even if your account was registered to a minor, you could have very explicit role-play chats and you could also have those via the voice tool inside of uh you know what Meta calls its AI studio and Meta had licensed a bunch of celebrity voices. So, while Meta told me, hey, you know, this, as far as we can tell, this happened, you know, very, very rarely, but it was at least possible for a minor to get in there and have sexually explicit roleplay with the voice of John Cena or the voice of Kristen Bell, even though the actor's contracts with Meta, according to Horwitz, explicitly prohibited this sort of thing. Right. So, how does this tie into the Open AI story? Well, what is so compelling about these bots? Again, it's they're telling these young people what they want to hear. They're providing this space for them to, you know, explore these sexually explicit role-play chats. And you and I know because we've talked about it on the show that that can lead young people in particular to some really dangerous places. Yeah. I mean, that was the whole issue with the character AI. Um, you know, tragedy, the the the 14-year-old boy who, um, died by suicide after sort of falling in love with this chatbot character. Um, but it's also just really gross. You could basically bait the chatbot into talking about um, you know, statutory rape and things like that. And it's just like like the the thing that bothered me most about it was that there appeared to have been conversations within Meta about whether to allow this kind of thing. And for explicitly this sort of engagement maxing reason, uh Mark Zuckerberg and other Facebook executives uh according to this story had argued uh to relax some of the guard rails around sexually explicit chats and roleplay uh because presumably when they looked at the numbers about what people were doing on these platforms with these AI chat bots and what they wanted to do more of it pointed them in that direction. Yes. And while I'm sure that Meta would deny that it removed those guard rails, um it did go, you know, in in the run-up to the publication of the journal story and add some new features in that is designed to prevent minors in particular from having these chats. But another thing happened this week, Kevin, which is that Mark Zuckerberg went on the podcast of Dwarvesh, uh Dwaresh, who recently came on hardfork, and Dwar asked him, "How do we make sure that people's relationships with bots remain healthy?" And I thought Zuckerberg's answer was so telling about what Meta is about to do. And I'd like to play a clip. The the average American, I think, has I think it's fewer than three friends. And the average person has demand for meaningfully more. I think it's like 15 friends. It it it you know, I think that there are all these things that are better about kind of physical connections when you can have them. But the reality is that people just don't have the connection and they feel more now, I think that there are all these things that are better about kind of physical connections when you can have them. But the reality is that people just don't have the connection and they feel more alone um a lot of the time than they would like. So I agree with part of that and and I do think that bots can play a role in addressing loneliness. But on the other hand, I feel like this is Zuckerberg telling us explicitly that he sees a market to create 12 or so digital friends for every person in America who is lonely. And he doesn't think it's bad. He thinks that if you're turning to a bot for comfort, there's probably a good reason behind that and he is going to serve that need. Our default path right now when it comes to designing and fine-tuning these AI systems points in the direction of optimizing for engagement just like we saw on social media um where you had these social networks that used to be about connecting you to your friends and family. Um, and then because there was this sort of growth mindset and this growth imperative and because they were sort of trying to maximize engagement at all costs, we saw kind of these these more um attentiongrabby short form video uh features coming in. We saw a shift away from people's uh real family and friends toward like influencers and professional content. And I just worry that the same types of people, or in Mark Zuckerberg's case, literally the same people who made those decisions about social media platforms that I think a lot of people would say have been pretty ruinous, um, are now in charge of tuning the chat bots that millions or even billions of people are going to be spending a lot of time with. Yes. My my feeling is if you are somebody who who was or is worried about screen time, I think that the chatbot phenomenon is going to make the screen time situation look quaint, right? Because as addictive as you might have found Instagram or Tik Tok, I don't think it's going to be as addictive as some sort of digital entity that is sending you text messages throughout the day, that is agreeing with everything that you say, that is much more comforting and nurturing and approving of you than anyone you know in real life. Like we are just on a glide path toward that being a major new feature of uh of life around the world. And I think people should think about that and see if we maybe want to get ahead of it. Yeah. And I think the stories we've been talking about so far about ChatGBT's new uh sort of sycopantic model and Meta's uh sort of unhinged AI chatbots, those are about things that self-identify as chat bots. People uh know that they are talking with an AI system and not another human. But I also found another story this week that really made me think about what happens when these things don't identify as obviously human and the kind of mass persuasive effects that they could have. This was a story uh that came out of 404 media about uh an experiment that was run on Reddit by a group of researchers from the University of Zurich um that used AI powered bots without labeling them as such to pose as users on the subreddit r/changemy view which is basically a run on Reddit by a group of researchers from the University of Zurich um that used AI powered bots without labeling them as such to pose as users on the subreddit r/changemy view which is basically a subreddit where people attempt to change each other's views or persuade each other of things that are counter to their own beliefs. And these researchers, according to this report, uh, created essentially a large number of bots and had them try to leave a bunch of comments posing as various people, including a black man who was opposed to Black Lives Matter, a male survivor of statutory rape, and essentially tried to get them to change the minds of real human users about various topics. Now, a lot of the conversation around this story has been about the ethics of this experiment, which I think we can all agree are are um somewhat suspect. Yes. Yes. This is not a a well-designed and ethically conducted experiment. But the conclusion of the paper uh this this paper that is now I guess not going to be published um was actually more interesting to me because what the researchers found was that their AI chat bots were more persuasive than humans and surpassed human performance substantially at persuading real human users on Reddit to change their views about something. Yeah. Yeah. So, the the way that this works is that if uh if a if a human user posts on change my view, like change my view about this thing, and then someone in the comments does successfully change their view, they award them a point called a delta. And these researchers were able to uh earn more than 130 deltas. Um, and I think that speaks to Kevin uh just what you've said that these things can be really persuasive in particular when when you don't know that you are talking to a bot. So, while the first part of this conversation is sort of about, you know, when you're talking to your own chatbot, could it maybe lead you astray? That's dangerous. But hey, at least you know you're talking to a chatbot. The Reddit story is the flip side of that, which is this reminder that already as you're interacting online, you may be sparring against an adversary who is more powerful than than most humans at persuading you. Yeah. And so Casey, if we could sort of tie these three stories together into a single, I don't know, topic sentence, what would that be? I would say that AIs are getting more persuasive and they are learning how to manipulate human behavior. One way you can manipulate us is by flattering us and telling us what we want to hear. Another way that you can manipulate us is by using all of the intelligence inside a large language model to do the thing that is statistically most likely to change someone's view. Kevin, we are in the very earliest days of it. But I think it's so important to tell people that because in a world where so many people continue to doubt whether AI can do almost anything at all, we've just given you three examples of AI doing some pretty strange and worrisome things out in the real world. Yes. And all of this is not to detract from what I think we both believe are the real benefits and utility of these AI systems. Not everyone is going to experience these…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.