Shawn Wang: belief

Transcript context

…Ouch. Yes. Yeah. But the thing I've promised myself is we will support all of them, regardless of what it costs. And like, once again, I see this as a just a research thing, you know, benefit to humanity and inference costs are going down. At least I so I tell myself late at night so I can sleep. So they pick the highest numbered one. And so we have an option in there right now that says, which is the first option. It's like, let the system pick for me. Auto-optimist. Yeah. As it turns out, people don't do that. They just pick the, because they don't trust it yet, which is fine. They shouldn't trust it completely. But one thing we discovered is that if we back channel it, and this is the thing we're testing with, is that, oh, if I can just run the exact same agent that gets run a thousand times, we'll do it on our own internal agents first. And if the ratings and reviews, because we're getting human evals all the time on these agents, we can get a dramatic multiple orders of magnitude reduction by going to a lower model with literally like no change in the quality of the output. Right. Which makes sense. Because so many of the things we're doing doesn't require the most powerful model. And it's actually bad because there is higher latency. It's not just a cost thing. But so anyway, like in that kind of future state, I think we're going to have model routing and a whole body of people working on that problem, too. It's like, help me pick the best model at runtime. Would you buy or build model routing? I buy everything that I can buy. I don't want to build anything if I don't have to. One of the most impressive examples of this. I think was our Chai AI conversation, which I think about a lot. He views himself explicitly as a marketplace. You are kind of a marketplace, but he has a third angle, which is the model providers, and he lets them compete. And I think that sort of Chai three-way marketplace maybe makes a lot of sense. Like, I don't know why every AI company isn't built that way. It's a good point, actually. Yeah, it makes sense. I have a list of things I'm super passionate about. I'm very passionate about efficient markets or extremely irritated by inefficient markets. And so efficient markets, for the normies listening, are markets that exist where every possible efficient markets are the ones that every transaction that should occur actually does. That's an efficient market that should happen. And so then why do inefficient markets exist? Well, maybe the buyer and seller don't know about each other. Maybe there's not enough of a trust mechanism. There's no way to actually price that or come up with fair market value for fair pricing. And as you kind of knock those dominoes down, the market becomes more and more. And lots of latent value exists as a result of inefficiency. And whoever removes those inefficiencies. Yeah. And then the market recedes for high value markets makes a lot of money. That's been proven time and time again. This is one of those examples of there's an inefficiency right now because we are either over using over models or whatever. Let's just reduce that to an efficient market. The right model should be matched up with the right use case for the right price. And then we'll... Very interesting. You ever looked into DSPy? I have looked at it. Not deeply enough, though.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.