Sholto Douglas: prediction

Transcript context

…Just flip the sign. Just flip the sign. And so, once you get to the stage where your models are capable of implementing one of Noam's ideas, and then you can just let them loose and let them build that intuition of how to do scientific discovery. The key thing here, again, is the feedback loops of. I expect scientific areas where you are able to put it in a feedback loop to have, eventually, superhuman performance. One prediction I have is that we're going to move away from “can an agent do XYZ”, and more towards “can I efficiently deploy, launch 100 agents and then give them the feedback they need, and even just be able to easily verify what they're up to” right? There's this generator verifier gap that people talk about where it's much easier to check something than it is to produce the solution on your own. But it's very plausible to me, we'll be at the point where it's so easy to generate with these agents that the bottleneck is actually, can I as the human verify the answer? And again, you're guaranteed to get an answer with these things. So, ideally, you have some automated way to evaluate and test a score for how well it worked, how well did this thing generalize? And at a minimum, you have a way to easily summarize what a bunch of agents are finding. It's like, okay, well if 20 of my 100 agents all found this one thing, then it has a higher chance of being true.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.