Demis Hassabis: evaluation

Transcript context

…Do you think the scaling laws are holding strong on the pre-training/post-training test time compute? Do you on the flip side of that, anticipate AI progress hitting a wall? We certainly feel there’s a lot more room just in the scaling. So actually all steps pre-training, post-training, and inference time. So there’s sort of three scalings that are happening concurrently. And again there, it’s about how innovative you can be and we pride ourselves on having the broadest and deepest research bench. We have amazing, incredible researchers and people like Noam Shazir who came up with Transformers and Dave Silver who led the AlphaGo project and so on. And that research base means that if some new breakthrough is required, like an AlphaGo or Transformers, I would back us to be the place that does that. So I’m actually quite like it when the terrain gets harder, right? Because then it veers more from just engineering to true research, and research plus engineering, and that’s our sweet spot and I think that’s harder. It’s harder to invent things than to fast follow. And so we don’t know, I would say it’s kind of 50/50 whether new things are needed or whether the scaling the existing stuff is going to be enough. And so, in true kind of empirical fashion, we are pushing both of those as hard as possible. The new blue sky, ideas and maybe about half our resources are on that. And then scaling to the max, the current capabilities. And we’re still seeing some fantastic progress on each different version of Gemini. That’s interesting the way you put it in terms of the deep bench, that if progress towards AGI is more than just scaling compute, so the engineering side of the problem, and is more on the scientific side where there’s breakthroughs needed, then you feel confident DeepMind as well, Google DeepMind as well positioned to kick ass in that domain.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.