Chip Huyen: evaluation

Transcript context

…When you say data preparation, what's an example to make that real and concrete for us to understand? So one way is mentioned as in you have chunks of data. So we have think about how big of each chunk should be. Because if it's sort of think about it's a context you want to maximize, maybe you can, it's very simple example. You want to retrieve a thousand words. So if a data chunk is long, then it's more likely to contain more relevant metadata so it can retrieve more. But if it's too long then you have a thousand word. And so chunk is like a thousand words, you can reach one chunk. So it's not very useful. But if it's too short, then you can retrieve more relevant information also. It can retrieve a wider range of documents and chunks, but at the same time each chunk is too small to contain relevant information. So we have very nice chunk design, how big each chunk should be. You add contextual informations like summary, metadata, hypothetical questions. Somebody was telling me just a very big performance they got is that from rewriting their data in the question-answering format. Instead of having... So they have a podcast instead of just chunking the podcast, you just reframe, rewrite it into here's a question, here's answers and produce a lot of them. It can use AI for that as well. So that's one example of data processing. A lot of example I see is for people helping, using AI to help specific [inaudible 00:36:40] use and documentations. And we write documentation. Usually a lot of documentation today is written for human reading and AI reading is different because it's different because humans, we have common sense and we kind of know what it is. So one things are, even for human experts, they have the context that AI doesn't quite have. So somebody told me that what's a big change they have is let's say, that you have a function. The documentation for this, maybe the library. As a library said okay, the output of this one is maybe talking for, I don't know, some crazy term, maybe some temperature or something on the graph. It should be like one zero or minus one. And as a human expert maybe understand the scale, what one in the scale mean, but for AI, just really doesn't understand what that means. So actually, have another annotation layer for AI. It's like, okay, good temperatures equal one means like that. It's not like it's a actual temperature. It's associated with the scale over there. So just saving all this data processing to make it easier for AI to retrieve the relevant information to answer the questions. This episode is brought to you by Persona, the verified identity platform, helping organizations onboard users fight fraud and build trust. We talk a lot on this podcast about the amazing advances in AI, but this can be a double-edged sword. For every wow moment, there are fraudsters using the same tech to wreak havoc, laundering money, taking over employee identities and impersonating businesses. Persona helps combat these threats with automated user business and employee verification. Whether you're looking to catch candidate fraud, meet age restrictions or keep your platform safe, Persona helps you verify users in a way that's tailored to your specific needs. Best of all, Persona makes it easy to know who you're dealing with without adding friction for good users. This is why leading platforms like Etsy, LinkedIn, Square and Lyft trust Persona to secure their platform. Persona is also offering my listeners 500 free services per month for one full year. Just head to withpersona.com/lenny to get started. That's withpersona.com/lenny. Thanks again to Persona for sponsoring this episode. Awesome. Okay. So you've talked a bit about how you work with companies on these sorts of things, on their AI strategies, on their AI products, how they build, which tools they build, all these things. I want to spend a little time here because a lot of companies are building AI products. A lot of companies are not having a good time building AI products. Let me ask a few questions along these lines of what you've learned working with companies that are doing this well. One is just, I guess, in terms of AI tool adoption and adoption in general within companies, there's all this talk recently of just all this AI hype. The data is actually showing most companies try it. Doesn't do a lot, they stop. And so there's all this just maybe this isn't going anywhere. So in terms of just adoption of tools in AI within companies, what are you seeing there?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.