Lenny Rachitsky: belief

Transcript context

…I think you're definitely going to live in a world where you have researchers built into every product team. And I don't even mean just at foundation model companies because I think the future... Actually, frankly one thing that I'm sort of surprised about about our industry in general is that there's not a greater use of fine-tuned models. A lot of people... These models are very good, so our API does a lot of things really well, but when you have particular use cases, you can always make the model perform better on a particular use case by fine-tuning it. It's probably just a matter of time. Folks aren't quite comfortable yet with doing that in every case. But to me, there's no question that that's the future. Models are going to be everywhere just like transistors are everywhere, AI is going to be just a part of the fabric of everything we do, but I think there are going to be a lot of fine-tuned models because why would you not want to more specifically customize a model against a particular use case? And so I think you're going to want sort of quasi researcher machine learning engineer types as part of pretty much every team because fine-tuning a model is just going to be part of the core workflow for building most products. So that's one change that maybe you're starting to see at foundation model companies that will propagate out to more teams over time. I'm curious if there's a concrete example that makes that real, and I'll share one that comes to mind as you talk, which is, when you look at Cursor and Windsurf, something I learned from those founders is that they use a Sonnet, but then they also have a bunch of custom models that help along the edges that make the specific experience that's not just generating code even better like auto-complete and looking ahead to where things are going. So is that one or any other examples of which you... What is a fine-tuned model? Do you think teams will be building with these researchers on their teams? Yeah. I mean, so when you're a model, you're basically giving the model a bunch of examples of the kinds of things you want it to be better at. So it's, "Here's a problem, here's a good answer. Here's a problem, here's a good answer," Or, "Here's a question, here's a good answer times a thousand or 10,000." And suddenly you're teaching the model to be much better than it was out of the gate at that particular thing. We use it everywhere internally. We use ensembles of models much more internally than people might think. So it's not, "I have 10 different problems. I'll just ask baseline GPT four oh about a bunch of these things." If we have 10 different problems, we might solve them using 20 different model calls, some of which are using specialized fine-tuned models, they're using models of different sizes because maybe you have different latency requirements or cost requirements for different questions. They are probably using custom prompts for each one. Basically you want to teach the model to be really good at... You want to break the problem down into more specific tasks versus some broader set of high level tasks. And then you can use models very specifically to get very good at each individual thing. And then you have an ensemble that tackles the whole thing. I think a lot of good companies are doing that today. I still see a lot of companies giving the model single, generic, broad problems versus breaking the problem down, and I think there will be more breaking the problem down using specific models for specific things, including fine tuning.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.