Sam Altman: prediction

Transcript context

…Yes, by how many people talk to AI. LLM psychosis, everyone on Twitter today is saying it’s a thing. How much a thing is it? A very tiny thing, but not a zero thing, which is why we pissed off the whole user base, or most of the user base, by putting a bunch of restrictions in place. The “treat adult users like adults” includes an asterisk, which is “treat adults of sound mind like adults.” Society decides that we treat adults that are having a psychiatric crisis differently than other adults. This is one of these things that you learn as you go, but when we saw, the kind of like put ChatGPT into role-playing mode, or pretend like it’s writing a book, and have it encourage someone in delusional thoughts, 99 point some big number percentage of adults, totally fine with that, some tiny percentage of people, also, if they talk to another person who encourages delusion, it’s bad. We made a bunch of changes, which are in conflict with the freedom of expression policy. Now that we have those mental health mitigations in place, we’ll again allow some of that stuff in creative mode, role-playing mode, writing mode, whatever, of ChatGPT. The thing I worry about is not that there will be a few basis points of people that are close to losing grips with reality, and we can trigger a psychotic break, and we can get that right. The thing I worry about more — it’s funny, the things that stick in your mind. Someone said to me once, “Never ever let yourself believe that propaganda doesn’t work on you. They just haven’t found the right thing for you yet.” Again, I have no doubt that we can’t address the clear cases of people near a psychotic break. For all of the talk about AI safety, I would divide most AI thinkers into these two camps of “Okay, it’s the bad guy uses AI to cause a lot of harm,” or it’s, “the AI itself is misaligned, wakes up, whatever, intentionally takes over the world. ” There’s this other category, third category, that gets very little talk, that I think is much scarier and more interesting, which is the AI models accidentally take over the world. It’s not that they’re going to induce psychosis in you, but if you have the whole world talking to this one model, it’s not with any intentionality, but just as it learns from the world in this continually coevolving process, it just subtly convinces you of something. No intention, it just does. It learned that somehow. That’s not as theatrical as chatbot psychosis, obviously, but I do think about that a lot. Maybe I’m not good enough, but as a professor, I find people pretty hard to persuade, actually. I worry about this less than many of my AI-related friends do.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.