Dylan Patel: uncertainty

Transcript context

…Well, what’s your prediction there? Because my gut says we’re very far away from that. I think OpenAI’s statement, I don’t know if you’ve seen the five levels where it’s chat is level one, reasoning is level two, and then agents is level three. And I think there’s a couple more levels, but it’s important to note, we were in chat for a couple years. We just theoretically got to reasoning, we’ll be here for a year or two, and then agents, but at the same time, people can try and approximate capabilities of the next level, but the agents are doing things autonomously, doing things for minutes at a time, hours at a time, et cetera, right? Reasoning is doing things for tens of seconds at a time and then coming back with an output that I still need to verify and use and try check out. And the biggest problem is of course, it’s the same thing with manufacturing. There’s the whole six sigma thing, how many nines do you get? And then you compound the nines onto each other and it’s like if you multiply by the number of steps that are six sigma, you get to a yield or something. So in semiconductor manufacturing, tens of thousands of steps, 9999999 is not enough. You multiply by that many times you actually end up with 60% yield, right? Really low yield or zero. And this is the same thing with agents, right? Chaining tasks together each time, even the best LLMs in particularly pretty good benchmarks don’t get 100%, right? They get a little bit below that because there is a lot of noise. And so how do you get to enough nines, right? This is the same thing with self-driving. We can’t have self-driving because without it being super geofenced like Google’s and even then they have a bunch of teleoperators to make sure it doesn’t get stuck. But you can’t do that because it doesn’t have enough nines. Self-driving has quite a lot of structure because roads have rules, it’s well-defined, there’s regulation. When you’re talking about computer use for the open web, for example, or the open operating system, it’s a mess. So the possibility… I’m always skeptical of any system that is tasked with interacting with the human world, with the open messaging world.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.