Logan Kilpatrick: belief

Transcript context

…Yeah, this is such an interesting space, and I think it's another space where I'm excited for people to do more scientific empirical studies about, because there's so much gut feeling, best practices that maybe aren't actually true in a certain way. I think the reason that prompt engineering exists and comes up at all is because the models are so inclined, because of the way that they're trained, to give you just an answer to the question that you ask. Crap in crap out. If you ask a pretty basic question, you're going to get a pretty basic response. And actually the same thing is true for humans, and you can think of a great example of this. When I go to another human and I ask, "How's your day going," they say, "It's going pretty good." Literally, absolutely zero detail, no nuance, not very interesting at all versus, again, if you have some context with a person, if you have a personal relationship with them and I ask you, "Hey Lenny, how's your day going? How did the last podcast go," et cetera, et cetera, you just have a little bit more context and agency to go and answer my question. I think this is prompt engineering. My whole position on this is prompt engineering is a very human thing. When we want to get some value out of a human, we do this prompt engineering. We try to effectively communicate with that human in order to get the best output. And the same thing is true of models. And I think it's like, again, because we're using a system that appears to be really smart, we assume that it has all this context, but it's really like imagine a human level intelligence but literally no context. It has no idea what you're going to ask it. It's never met you before. It has no idea who you are, what you do, what your goals are. And it's the reason that you get super generic responses sometimes is because people forget they need to put that context in the model. So I think the thing that is going to help solve this problem, and we already kind of do this in the context of Dali, so when you go to the image generation model that we have, Dali, and you say, "I want a picture of a turtle," what it does is it actually takes that description. It says, "I want a picture of a turtle," and it changes it into this high fidelity, like generate a picture of a turtle with a shell, with a green background and lily pads in the water and all this other. It adds all this fidelity because that's the way that the model is trained. It's trained on examples with super high fidelity. This will happen with text models. You can imagine a world where you go into ChatGPT and you say, "Write me a blog post about AI." model is trained. It's trained on examples with super high fidelity. This will happen with text models. You can imagine a world where you go into ChatGPT and you say, "Write me a blog post about AI." It automatically will go and be like, "Let me generate a much higher fidelity description of what this person really wants, which is generate me a blog post about AI that talks about the trade-offs between these different techniques and some example use cases and references some of the latest papers," and it does all that for you, and then you at the user will hopefully be able to be like, "Yep, this is kind of what I wanted. Let me edit this. Let me edit this here." And again, the inherent problem is we're lazy as humans. We don't want to type all... We don't really want to type what we mean, and I think AI systems are actually going to help solve some of that problem. So until that day, what can people do better when they're prompting, say ChatGPT? And I'll give you an example. Tim Ferris suggested this really good idea that I've been stealing, which is when you're preparing for an interview, you go to chat GPT. And so I did this for you. I was like, "Hey, I'm interviewing Logan Kilpatrick, he is head of developer relations at OpenAI, on my podcast. Give me 10 questions to ask him in the style of Tyler Cowen," who I think is the best interviewer. He is so good at just very pointed original questions. So what advice would you have for me to improve on that prompt to have better results? The questions were fine. They're great. They're interesting enough, but they weren't like, "Holy, these are incredible." So I guess what advice would you give me in that example?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.