Ankur Goyal: disagreement

Transcript context

…Right now it's a podcast network by Eric Tornberg. Yes, that's actually why I thought of that word. Old-school elbow grease is what I'm saying, of hand-tuning prompts, that's another way of achieving that business goal. And there's actually a lot of cases where hand-tuning a prompt performs better than fine-tuning because you don't accidentally destroy the generality that is built into the world-class models. So in some ways it's safer, right? But really the goal is automatic optimization. And I think automatic optimization is a really valid goal, but I don't think fine-tuning is the only way to achieve it. And so, in my mind, for it to be a business, you need to align with the problem, not the technology. And I think that automatic optimization is a really great business problem to solve. And I think if you're too fixated on fine-tuning as the solution to that problem, then you're very vulnerable to technological shifts. There's a lot of cases now, especially with large context models, where in-context learning just beats fine-tuning. And the argument is sometimes, well, yes, you can get as good a performance as in-context learning, but it's faster or cheaper or whatever. That's a much weaker argument than, oh my god, I can really improve the quality of this use case with fine-tuning. It's somewhat tumultuous. A new model might come out, it might be good enough that you don't need to use, or it might not have fine-tuning, or it might be good enough that you don't need to use fine-tuning as the mechanism to achieve automatic optimization with the model. But automatic optimization is a thing. And so that's kind of the semantic thing, which I would say is maybe, at least to me, it feels like more of an absolute. I just don't think fine-tuning is a business outcome. There are several means to an end, and the end is valuable. Now, is fine-tuning a technically valid way of doing automatic optimization? I think it's very context-dependent. I will say, in my own experience with customers, as of the recording date today, which is September or something, very few of our customers are currently fine-tuning models. And I think a very, very small fraction of them are running fine-tuned models in production. More of them were running fine-tuned models six months ago than they are right now. And that may change. I think what OpenAI is doing with basically making it free and how powerful Llama 3 AB is and some other stuff, that may change. Maybe by the time this airs, more of our customers are fine-tuning stuff. But it's changing all the time. But all of them want to do automatic optimization. Yeah, it's worth asking a follow-up question on that. Who's doing that today well that you would call out?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.