Logan Kilpatrick: belief

Transcript context

…I think the other thing I would say as far as lessons learned is there's really no substitute for being able to just experiment and iterate quickly. So I think this goes to all of Logan's points about the foundation being strong, but I really think what has helped us is really being able to put in, for example, a new model iterate really quickly with a product on like, hey, what are the right prompts that would actually make this model viable for a different situation? What are the ways to kind of prototype really quickly with this model? What are the ways to get it in the hands of even just internal users quickly, let alone external users? And I think that is something that is now more and more possible with kind of layers that are consistent across the team. I think it's pretty amazing to see the speed at which we can go from having a checkpoint that we're really excited about to putting it in the hands of internal developers to then seeing it come to life in a product. And then only when you see it come to life in the product do you really start finding its rough edges and to be able to actually then kind of come to terms with how you do that. And so more and more than it becomes like, okay, how do you have the right ability to tune prompts quickly? How do you have the ability to run really good live experiments where you can get really good data and feedback quickly? How can you build evals that help give you real signal? Those are the things that will speed up your progress of quality the most because it will give you the ability to actually get to the kind of product that you love. And I think if you think about NotebookLM, I mean, that team really understands the model. Like they are just like, I mean, you talk about a Banger product, it comes from like a Banger team. Like they are really good at being able to take the model and play with it quickly and prototype quickly to get to something amazing. And I think you see that actually play out in the product. The best example of this is the original sort of audio overview experience. And I think the thing that like shocked people about audio overviews was like the coherence of the dialogue. And the coherence of the dialogue was just base Gemini with a bunch of Banger prompts. And they sort of like knew how to sort of prompt whisper the model and get the best out of it. I think obviously the model that the actual audio model was really good as well, but like the prompt dialogue was really difficult for them to pull off and they pulled it off in an incredible way and I think helped people fall in love with that product. So it sounds like one big lesson is kind of modularizing. It used to be sort of the model on one side and then like everything else that goes into the product on the other side. And we're pulling a lot of the surrounding code and architecture and tools onto the model side.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.