Dwarkesh Patel: preference

Transcript context

…That’s distillation, an explanation. If I’m thinking of your quality as an essay writer—if I give you a book to read and I want a book report—I might believe that the LLM gives me a better book report. But what people are really getting at when they say it’s bad is, what is writing? It’s not just the distillation of preexisting ideas. It’s not just how you explain clearly, because they are good explainers. It’s about what the insight is. This is where autoregression is a very weird way to generate things. When you’re writing, you sort of know that in order for it to be good, you have to have an element of the unpredictable. It’s not just increasing the temperature in your mind. It’s knowing exactly the correct point when you want to make an unpredictable move, and that that’s going to be what’s more insightful. Even if it’s better at explaining a preexisting thing, what generated that book that you wanted distilled in the first place? It wasn’t an LLM that generated it and you just needed it. It was some author who, through a lot of exploration of ideas in the world, decided what aspects were interesting and what ways of presenting it formed a coherent, well-motivated narrative. They put that all together in some way. If they’re a good author, you would probably err on the side of reading their book instead of the distillation. Still, what makes it worthwhile to explore at all in the first place and want to upload it at all? It’s that side of it that people cite when they say LLMs are bad at writing. It’s that element of unpredictability, of deliberately choosing something novel that is very directly contradictory to the way things are typically produced. That’s a good point. I think they’re also really bad at building really good mental models of people, which is a very important skill in writing. Andy Matuschak and another collaborator, whose name I’m forgetting right now, did an interesting report where they tried to teach LLMs to write good spaced-repetition prompts. I really like this because even though it seems like a totally random skill… It’s just like, people are talking about recursive self-improvement in a year, and we can’t get these things to write good flashcards. What’s going on there? They tried many different kinds of techniques, and they’re sophisticated people. They tried to RL open source models. They tried all kinds of things, including chain of thought and a big prompt they sent to the best closed source model. The key constraint, it seemed to me, was that writing a good card is about projecting somebody’s mind in three months. What is the way in which they’ll associate the question? What kind of answer will they be thinking at that moment? Is the elicitation that inspires the detail you actually want to take away from the passage you’re trying to make cards about? I think writing is similar to this. If you’re writing something, the reason it’s such an enervating process that takes so long is that with each word or each sentence, you have to be thinking: what is happening in my reader’s mind right now? Even if I flip the phrasing around so the end phrase goes to the beginning and this is the first image that comes to your mind before you read the rest of the sentence… Maybe autoregression is bad at that. This is maybe a more diffusion-like property of considering the whole rather than going sentence by sentence. But also I think that requires a lot of mentalizing, which these models weirdly struggle at. It’s an interesting question. Is it weird that they struggle at that? I might butcher this. You know how you cite studies that you once read and maybe the study wasn’t real? There’s one very memorable one. Let’s say you want to quiz people’s EQ. You show a flashcard of someone’s facial expression and someone is trying to describe that emotion. There are really good tests online that have a face and then four possible emotions. It’s surprisingly hard to describe exactly the correct emotion, but you also get the sense there really is a correct answer. If you try this with people in your life, you’ll notice that the ones who are pretty plugged in socially do really well on it, and the ones who are a little bit more left-brain don’t. That is a kind of test you can do. I vaguely remember an experiment to this effect where they took people who had freshly gotten Botox, and they did a pretest and a post-test. Post-test, they were just much worse at reading people’s expressions. That feels weird.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.