Daniel Miessler: evaluation

Transcript context

…I guess one thing that obviously we should give the sort of skeptic their due, at least in terms of one follow-up question there. Why hasn't that happened more than it has already? And I'll confess that I'm not a super forecaster, but I have done some of these like forecasting exercises where a year ago I predicted a bunch of stuff where it was going to be today. And I would say I always overestimate how much disruption, at least over the last three, four years. I think I've consistently overestimated how much disruption we would see in the next year. I think I've had a better sense of like where the capabilities would go. Probably overestimated that a little bit as well, but not much. But I've much more so overestimated like how different will the world be a year from now. Yeah. that? Yeah, it's a great question. I make sense of it because I think the value of AI is actually Actually, in the scaffolding more so than the models. So, So what the model is capable is capable of. of doing doesn't really matter if it's not inside of a scaffold that allows it to take inputs and produce outputs that are actually useful. This is why Cloud Code has gone crazy because it is the best scaffolding system, right? The difference between Opus 4.5 and the best Gemini model or the best OpenAI model is not much. And the other two are better in some ways. In fact, the open source models are very close. It's not Anthropic that's blowing up. It's not Opus 4.5 that's blowing up. It's Cloud Code because it's scaffolding. And to answer your question about why this hasn't happened before, even before AI, but in the previous three years of AI, it's because the, in my mind, it's because the average knowledge worker job is extremely general. So when they come into work, you've got to check all these emails. Oh, but you have to watch this video because it's mandatory secure code training. Oh, but also there's this fight going on with your boss and this other person, and you've got to talk about that. Oh, it turns out you have to have an HR meeting. Oh, actually, corporate goals just changed completely. Now we have to redo all of our work. So we're not working on that project anymore. We're spinning over to this other project. So in the course of a week or a month or a year, human workers are being asked to do like these vastly different things. Even in the course of an hour, you might have to check emails, you might have to fix your email, you might have to watch a training course. There's not a scaffolding system that exists right now that would allow an AI to do all of that. It just wouldn't be possible. So you would have AI that's really good at the coding part. Maybe it's really good at writing reports. But how is it taking all those inputs in and producing the output in the same way that a human worker can? It can't, right? And that's why we don't have like giant armies of AI employees out on the market yet. And here's what I'm very worried about. And this is why I think 2027 is the year for AGI in my definition, which is the ability to replace an average human knowledge worker, right? The question is: when will it be, when will the scaffold? And we just saw co-work come out. Is that what they called it? Anthropic co-work? Yeah. We just saw that come out. That is a scaffold system for doing broad tasks at work, right? It's actually for more general tasks as well. But that is the type of thing that somebody can build an AI product on that actually replaces human workers. Because now all those weird, general things that are happening inside the company, those are just one-off tasks. And here's a really crucial point here: it doesn't matter for the replacement of human work and the disruption of the labor economy. hat are happening inside the company, those are just one-off tasks. And here's a really crucial point here: it doesn't matter for the replacement of human work and the disruption of the labor economy. It doesn't matter if it happens with the wizard behind the curtain, which is actually doing a whole bunch of narrow AI, but it's able to do it for all the tasks that an average worker does. And it's just being handled seamlessly with the scaffolding. It doesn't matter if it actually does it way better than an average employee. So when I talk about AGI, I'm not talking about what Archive thinks, the technical research papers. I think it's cool that they're going down that path. And I can't wait to see what they do if they create a truly AGI ASI intelligence. But what I care about is the humans. I care about who's getting fired, who's getting not hired. And I think the way that happens is through a scaffold that can actually do their work better than them, which I think is going to look a whole lot like Pi, which is the project I'm doing, Claude Code, which is what Pi is built on, and Cowork, which they just built with Claude Code. They said they've built it in a week, and there were no humans involved. Cloud Code wrote all the…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.