Yi Tay: belief

Transcript context

…But you will use those sparingly because they're expensive calls. Yeah, you, it depends on like the nature of the, the, the application, I think. Because if in rac, right, like you, there's a lot of issues like, okay, how you, like, you, the, the retrieval itself is the issue. Or like, you know, you, you, you might get fragmented if it's like, what if it's like a very complex story, right? That you like a storybook or like a complex like thing, right? And then, and then like we, like rec is very like, you kind of chunks, chunks and chunks, right? Yeah. The chunking is like, and you definitely have lots of information, right? So there I, there are a lot of application use cases where you just want. The model is like you were like, okay, like a hundred bucks, like take your time, take one whole day, come back to me with like an answer, right? Rather than like, I pay like, like one cent and then like get back a wrong answer. So I think that's like, that is actually very easy to show that RAC is better than long context because there are a lot of tasks that don't need this long context. You like, like fact retrieval, you just like RAC and then you do this thing, right? So like, long context may get a unfairly bad rap sometimes because like it's very easy to show like, RAC is like, 100 times cheaper, and it's very easy to show this, right? But then it's also, like, not so easy to emphasize the times where you actually really need, like, the long context to really make, like, very, very, very, very, very good, like, decisions. So, yeah, I mean, I think both have pros and cons depending on the use cases. Using them together is also interesting. hyperparameter that you have to wiggle around, right? Yeah. There's another wiggle on the hyperparameter, or there's another fog on the hyperparameter, which is how much you fine tune. New knowledge into the model. Are you positive on that? Do you have any views? So, for example, instead of doing RAG on a corpus and then inserting it into context, you would just fine tune your model on the corpus, so it learns the new knowledge. In whatever capacity,…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.