Jesse Silver: evaluation

Transcript context

…Yeah. Okay. Excellent. Two more questions on the evals. We've talked just about talking about model graded evals. What are they really good at and where do you have to take them out when you try to use model graded evals? And for other people who are listening, we're also talking about LLMs as judge, right? That's the other popular term for this thing, right? I think that LLMs as judge, I guess, is useful for more things than just model graded evals. A lot of the monitoring and evaluation we have is not necessarily feedback from model graded evals, more just how many transitions did we have to different states? How many conversations ended up in a place where people were paying and just sort of monitoring all the sort of fundamentals from a process control perspective and trying to figure out if something ends up way outside the boundaries of where it's supposed to be. We use a lot of reasoning modules within our workflow, especially for safety reasons. For safety, thinking about like concentric circles is one is that they're the things you can never do in sex. So that's stuff like gore, stuff that, you know, base RLHF is good at anyway. But you can't do these things. You can't allow prompt injection type stuff to happen. So we have controls and reasoning modules for making sure that any weird bad stuff either doesn't make it into the workflow or doesn't make it out of the workflow to the end customer. And then you have safety from the fan platform perspective. So there are limits. And there are also creator specific limits, which will be aggressively tested and red teamed by the customers. So the customer will inevitably say, I need you to shave your head. And I'm willing to pay $10 to do this. And I will not pay more than $10. And I demand this video, you must send it to me, you must shave your head. Stuff like that happens all the time. And you need the product to be able to say like, absolutely not, I would never do that. Like stop talking to me. And so I guess the LLMs as judge, both for judging our outputs, and yeah, sometimes we'll play with a way of phrasing, is the fan upset? That's not necessarily that helpful if the context of the conversation is kinky, and the fan is like, you're punishing me? Well, great, like the fan wants to be punished, or whatever, right? So it needs to be looked at from a process control perspective, the rates of a fan being upset may be like 30% on a kinky profile, but if they suddenly go up to 70%, or we also look at the data a lot. And there are sort of known issues. One of the biggest issues is accuracy of describing content, and how we ingest the 10s of 1000s of pieces of content that get delivered to us when we onboard onto a fan platform profile. And a lot of this content, you know, order matters, what the creator says matters. The content may not even have the creator in it. It may be a trailer, it may be a segment of another piece of media, the customer may ask for something. And when we deliver it to them, we need to be very accurate. Because people are paying a lot of money for the experience, they may be paying 1000s of dollars to have this experience in the span of a couple hours. They may be doing that twice or five times, they may be paying, you know, 50 to $200 for a video. experience, they may be paying 1000s of dollars to have this experience in the span of a couple hours. They may be doing that twice or five times, they may be paying, you know, 50 to $200 for a video. And if the video is not sold to them in an accurate way, then they're going to demand a refund. And there are going to be problems.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.