Raza Habib: evaluation

Transcript context

…you ever attempted to classify the use cases that you're seeing? Just, you see the whole universe, and you're not super opinionated about them. But, like, you know, there's summarization, there's classification, there's, you know, Okay, so interesting. I've not, I've certainly not tried to classify them as that granularity. Like, is it summarization or question answering? I often think more about the end use case. So, like, is this an edtech use case? Right, that's the vertical to me. I think, I think a little bit more about it like that. In terms of use cases, it's really varied, right? There are people using the models as completion, there's chat, like it wouldn't be so obvious to know without doing some like GPT level analysis on it, like getting GPT to look at the outputs and inputs, which you can do, which we can do. Whether they're doing summarization or something similar, but I would say I feel like most use cases blend like that to me feels like an old school N L P way of viewing the world. Like an old school N L P. We used to break down these tasks into like summarization and N E R and extraction and QA, and then pipeline things together. And actually I feel like. That doesn't map very well onto how well, how people are using TP4 today. Because they're using them as general purpose models. And so it is one model that's doing NER, it's doing extraction, it's doing summarization, it's doing classification, and it's often in one, one end to end sort of system. I think that's, that's what people want to believe, that they're using them as general purpose models. But actually when you open up the covers and look at the volume, 80 percent of it is. Some really dumb use…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.