Ryan Greenblatt: belief

Transcript context

…But stepping back, GPT-7.5 becomes GPT-8 as a result of all this AI R&D training and just generally becoming smarter. Then it helps you build GPT-9. Another very important thing has to happen, which is maybe the thing I’m most skeptical of. GPT-8 has figured out how to make it so… GPT-9, as intelligent as it is… Humans currently, AI researchers, try their stuff, and they’re like, “Okay, but we trained GPT-4.5 and it wasn’t good.” It required real-world feedback or some evaluation of trying to use the model in production. Then they were like, “It wasn’t that good, and we’re not going to ship it.” So GPT-8 needs this ability to see how good the transfer is to all these other things you’re talking about — like being really good at Texas politics, or really good at running a business, et cetera — which is not a production environment and, in fact, cannot be a containerized environment given the nature of the task. As the agents get longer and longer horizon, the short-horizon things you can containerize are like, “Okay, code this up or whatever.” Extremely long-horizon things — “Go run a successful business, go have a profitable day in the markets, go negotiate a trade deal” — these things are actually very hard to containerize. So I think it’s very plausible that it’s very hard for GPT-8 to figure out how to make this transfer to those environments. It may just not be in the nature of the training. Or maybe by default, training just doesn’t generalize in that way. So a concern you might have is: we train GPT-8, and GPT-8 is again better at all the R&D tasks that we can measure but is not good at some downstream tasks we care about. I have a few points. First, I expect that if you do the obvious thing, you will get pretty good transfer. You’ll be able to hold out some of the obvious stuff you’re doing. When I say “do the obvious thing”, I just mean training on a wide variety of different environments where the AI has to accomplish weird objectives in all kinds of different cases and learn about what’s going on. The second point is you’ll be able to get some feedback with some environments. You can get a sense of what it can do over the course of a few days in various different contexts. If it’s transferring to really out-of-distribution things, like doing some weird task in a few days in the real world, maybe you think it’s also transferring to doing things over a longer time period or whatever. I think the details of that vary though. The third thing is that for the world to be radically transformed, it is sufficient for the AIs to be really good at R&D. If the AIs were really, really good at chip R&D, building fabs, orchestrating factories, designing robots, operating robots, and also at AI R&D — developing AIs for new downstream domains with whatever data is available — I think that would already be a pretty crazy situation. From there, you can get what we might call an industrial explosion, where the AIs are building out way, way more compute. Also, maybe you’re already in a regime where AIs are doing huge amounts of R&D that humans have a hard time understanding. So the thing you’re pointing out is that there probably will be this transfer outside of these environments to maneuvering around in courtrooms and the halls of Congress and business boardrooms.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.