Ankur Goyal: belief

Transcript context

…I think I fully understand and mostly agree. I want to hear the opposite view. I think yours is not the consensus view, and I want to hear the other side. I mean, there's super smart people working on this, right? We'll be having Chroma and I think Qtrends on maybe Vespa, actually. One other part of the triangle that I drew that you disagree with, and I thought that was very insightful, was fine-tuning. So I had all these overlapping circles, and I think you agreed with most of them, and I was like, at the center of it all, because you need logging from Ops, and then you need a gateway, and then you need a database with a framework, or whatever, was fine-tuning. And you were like, fine-tuning is not a thing. It's not a business. So there's two things with fine-tuning. One is the technical merits, or whether fine-tuning is a relevant component of a lot of workloads. And I think that's actually quite debatable. The thing I would say is not debatable is whether or not fine-tuning is a business outcome or not. So let's think about the other components of your triangle. Ops slash observability, that is a business thing. Do I know how much money my app costs? Am I enforcing, or sorry, do I know if it's up or down? Do I know if someone complains? Can I retrieve the information about that? Frameworks, evals, databases, do I know if I changed my code? Did it break anything? Gateway, can I access this other model? Can I enforce some cost parameter on it? Whatever. Fine-tuning is a very compelling method that achieves an outcome. The outcome is not fine-tuning, it is can I automatically optimize my use case to perform better if I throw data at the problem? And fine-tuning is one of multiple ways to achieve that. I think the DSPY-style prompt optimization is another one. Turpentine, you know, just like tweaking prompts with wording and hand-crafting few-shot examples and running evals, that's another... Is Turpentine a framework? No, sorry, it's just a metaphor. But maybe it should be a framework. Right now it's a podcast network by Eric Tornberg.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.