Shawn Wang: belief

Transcript context

…My biggest question is how much can you differentiate at that model size? Like how big is going to be the performance gap between all these models? And are people going to be aware of what model is running? Right now for the large models, we're still pretty aware of like, oh, is this Sonnet 3.5, is this GPT-4, is this 3.145B. I think the smaller you get, the more it's just going to become like a utility. So you're not going to need a model router for small models. You're not going to need any of that. They're all going to converge to the best possible performance. Actually, Apple Intelligence is the model router, I think. They have something like 14, I did a count in my newsletter, like 14 to 20 adapters. And so based on your use case, they'll route and load the adapter or they'll route to OpenAI. So there is some routing there. To me, I think a lot of people were trying to puzzle out the strategic moves between OpenAI and Apple here because Apple is in a very good position to commoditize OpenAI. There were some rumors that Google was working with Apple to launch it. They did not make it for the launch. But presumably, Apple wants to commoditize OpenAI, right? So when you launch, you can choose your preferred external AI provider and it's either OpenAI or Google or someone else. That puts Apple at the center of the world with the ability to make routing decisions. I think that's probably good for privacy, probably good for the planet because you're not running oversized models on your spellcheck pass. I'm generally pretty positive on it. I'm not concerned about the capabilities issue. It meets their benchmarks. Apple put out a whole bunch of proprietary benchmarks because they don't like to do anything in the way that everyone else does it. So in the Apple Intelligence blog post, I think all of them were just their internal human evaluations and only one of them was an industry standard benchmark, which was IFEVL, which is good. But why didn't you also release your MMLU? Oh, because you suck on it. All right. I actually think all these models will be good. And on the Apple side, I'm curious to see what the price tag will be to be the default. Right now, Google pays them $20 billion to be the default search.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.