Demis Hassabis: preference

Transcript context

…So if we look at that AGI system, sorry to bring it back up, but AlphaEvolve, it’s super cool. So AlphaEvolve enables on the programming side, something like recursive self-improvement potentially. If you can imagine what that AGI system, maybe not the first version, but a few versions beyond that, what does that actually look like? Do you think it’ll be simple? Do you think it’ll be something like a self-improving- Like, do you think it’ll be simple? Do you think it’ll be something like a self-improving program and a simple one? I mean potentially that’s possible. I would say I’m not sure it’s even desirable because that’s a kind of hard takeoff scenario. But these current systems like Alpha Evolve, they have human in the loop deciding on various things, there’re separate hybrid systems that interact. One could imagine eventually doing that end to end. I don’t see why that wouldn’t be possible, but right now I think the systems are not good enough to do that in terms of coming up with the architecture of the code. And again, it’s a little bit reconnected to this idea of coming up with a new conjectural hypothesis, how they’re good if you give them very specific instructions about what you’re trying to do, but if you give them a very vague high level instruction, that wouldn’t work currently. And I think that’s related to this idea of invent a game as good as Go, right? Imagine that was the prompt. That’s pretty. And so the current systems wouldn’t know I think what to do with that, how to narrow that down to something tractable. And I think there’s similar, look, just make a better version of yourself. That’s too unconstrained. But we’ve done it. And as you know with AlphaVol, like things like faster matrix multiplication, so when you hone it down to very specific thing you want, it’s very good at incrementally improving that. But at the moment these are more incremental improvements, sort of small iterations. Whereas if you wanted a big leap in understanding, you’d need a much larger advance. Yeah. But it could also be sort of the pushback against hard takeoff scenario. It could be just a sequence of incremental improvements, like matrix multiplication. It has to sit there for days thinking how to incrementally improve a thing and it does solve recursively. And as you do more and more improvement, it’ll slow down. So there be, the path to AGI won’t be like a gradual improvement over time.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.