Nathan Labenz: belief

Transcript context

…Yeah, I would say, you know one of the things that being in search for a long time, search used to be short. It used to be, you probably don't remember back this far, but you used to type in two or three words to search and then it was longer. And then as there were other modalities of information being pushed to you, they kind of went shorter. Again, I think with large language models, what large language models do when they get a long query is they think, what is the set of queries that's going to help me answer this question? And then they fire off a set of queries and they're all quite long. If you watch Claude or Grok, they, they'll actually tell you in their tool calls. I know not everybody looks into them, but of course I do. If you look at them, they, they're alone. Sometimes you know, they're like 8 words and stuff. And I don't know if you can remember the last time you typed in eight words into a keyword search box, but definitely you'll notice in a large language model, it's almost like a full sentence or sometimes 2 sentences is good because you want to actually describe almost the essay that you want given back to you. So that is that how it has, you know, evolved. To contrast your approach, I think this is like very interesting and maybe the answer ultimately will be both. But when I think about a company like Exa and then your product, in some ways they're similar in that I think they're both kind of designed for AI users, right? The, the Exa paradigm is like you can write a whole paragraph and it's all very sort of semantically oriented, very embedding based. And but, but I've heard, I think I even spoke to will about the idea that, you know, nobody's going to type in a paragraph long query, but the, your AI can, you know, it has time to do that. And then you're taking kind of a different angle on the same thing saying, well, keyword and you can maybe tell us a little bit more about like how to think about how best to use a keyword based search. But it's, it's not semantic. It's it's not doing things like, you know, finding synonyms or, you know, doing like higher abstraction level embedding type matching. But the agent can, as you've said, kind of fire off dozens of these potentially to try to really cast a wide net. How do you think about the the kind of compare and contrast of those approaches? Do you think it will in the end, like all be using one of each at the same time? Or if if one paradigm wins out over the other, like why do you think one will win? What? What are the kind of, you know, drivers that would make one a better bet long term than the other? Well, I think AI is going to be picking the winners and and not us humans. And I think that of course real search engines do use things called stemming. If you say walk, then walking walked, all that are are very normal, which we have as well. We have some synonyms and we do process, you know, we do go through the corpus and process some semantic information. But then at runtime it is, you know ACPU plus GPU based system. It is not a vector database. I think that you know there's a number of things with vector databases. Google published a research paper about it that as you put more things in a vector database, now you're imagine you have a a multi billion space and you need to make a vector long enough to distinguish this one point in space. That vector to distinguish among billions of things starts getting longer. Now contrast that to 90% of web pages are less than 1K long if you're talking about number of words. So you know what a good representation of that point is the set of words on the page. So I do think that vector people and, and search people have a little different view. And the Google researchers think that vector DBS are great, but only scale to a certain amount. And so I think that's the challenge that they're going to be coming up against. There's two other challenges with vector databases. One, they are slower and then you know, the the last item is because they do a soft match, then sometimes relevancy can be a challenge. So that number of enterprise orgs that have used a vector database for RAG, now all of a sudden they have to turn into relevancy experts because they're like, why did this come back? And it's, it is because of those soft match features and on the shape of their corpus. So every enterprise doesn't really have the ability to all become relevant experts. So yeah, we are. The way we feel is that inside enterprises, if you use ceramic, we actually have a system that for that enterprise will actually tweak and learn a good ranking function and you just load it into the configuration and it's yours. So because not every query stream is the same, not every set of documents is the same. So I think that long term we're well positioned. But you know, EXA has done really well so far. So I like to say positive things about people.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.