Nathan Labenz: evaluation

Transcript context

…Yeah, great point. Sequence handles the full revenue workflow for complex pricing, from quoting and metering to invoicing, revenue recognition, and collections. Book a public demo at https://sequencehq.com and use code COGNISM in the source field to save 20% off year one Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr Can I dig in a little bit more on the core models that you guys are using to make predictions. And I'd love to understand the architecture of this better. I think for calibration, anybody who's listening to this feed is going to have a conversational familiarity at least with how large language models work. So we know that they're generating a token at a time. We know that the inputs get embedded and we know the kind of mechanics of the forward pass and all that stuff, right? And we know it's auto progressive, blah, blah, blah. This strikes me as a very different world. And I don't have nearly as much intuition for what the models are that are driving these things. I do know that they have to be a lot faster because the ad's gotta show up really quickly on the page. And then I know also that there's a pretty challenging matching problem in there somewhere because I've got millions of, you've got, we've got, society collectively has got millions of these profiles of individuals. And then also, as you said, into the tens of thousands of advertisers. And I don't know how much pre-computing is done or whatever, but it has to happen pretty quick on the load of a page. So could we kind of break down how big are these models? What do the inputs look like? You could imagine something very large and sort of very sparse set of inputs. But I guess it doesn't seem plausible, but it's like, here's all the websites, and here's which ones this user visited, right? That doesn't seem like it works. So there's got to be some sort of tokenization or something that is kind of bringing the user profile into a manageable state size so that it can be used as an input. I'm not even sure if I'm quite asking the right questions here. Tell me what this looks like under the hood. Yes, maybe I can take a quick stab at it and then Liva can take it down into more detail. So Liva actually referenced this earlier. So every single time that we want to, when we get an opportunity to show an ad, so that opportunity actually goes to multiple different ad tech providers who are all acting as kind of delegates on behalf of the actual advertiser themselves, whether it are brands or advertisers. And so the amount that we bid is based on how valuable that opportunity is to the advertiser. Effectively, how likely the user is to click on that ad and go back to the website and buy the product. And the way we evaluate that is we, through the mechanism we talked earlier, we see what products the users are interested in, what they've looked at, what they clicked through, what they've seen, what they buy, what they don't buy, and so on. As the display opportunity comes up, so we see the ID that we mentioned in the cookie, and then we take a look at all of the different products that that person has seen or whatever audience segments they belong to. And each one we can say, okay, based on all the different features we put into the model. So, you know, the products, the previous purchase history, the context of the website, the device they're on, a couple of other things that come in, and there's actually probably I'm not sure, it's like 150 different features we can take in. And each of those go into this calculating as part of this massive equation, which will tell us the likelihood that person is to click, the likelihood they are to click through to the website and eventually do a purchase. And all of that comes out to a value which we bid. If we win the opportunity, then we have to say, well, which products do we show and how do we do all of that kind of stuff? All of that process gets done in milliseconds because we use a lot of caching, we've trained the models offline, then the inference happens at real time in really low latency.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.