Paul Bloom: preference

Transcript context

…The people who are best at working with LLMs — what kind of psychological intelligence do you think they have? Who’s going to be good at that? I think you need a certain flexibility, a certain degree of patience. You need to develop a theory of mind with LLMs that is different from the theory of mind you need for people. I recently asked Claude — I use it a lot — I asked it, “There’s a phenomenon in children I found. Does the same phenomenon show up in adults?” I plainly framed the question that I wanted the answer to be yes. They said, “Yes, it is. Here are three papers that show it decisively.” None of the papers were real. It hallucinated it because it’s what I wanted. Then I went back and I said, “Look, I’ll lose my job if you give me papers that are false, that are made up. You’ve got to be accurate, or I’m in big trouble.” Then it did much better. You’ve got to game them to some extent, and in a very different way than you game a person. That capacity is something which I think is essential, at least, for the current models. Here’s a simple question. What’s the best test of whether someone or something has a theory of mind?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.