Carl Shulman: evaluation

Transcript context

…Okay, that's actually a really interesting point. Now somebody might say, there's not some sense in which AIs could universally speed up the progress of OpenAI by 50 percent or 100 percent or 200 percent if they're not able to do everything better than Ilya Sutskever can. There's going to be something in which we're bottlenecked by the human researchers and bottleneck effects dictate that the slowest moving part of the organization will be the one that kind of determines the speed of the progress of the whole organization or the whole project. Which means that unless you get to the point where you're doing everything and everybody in the organization can do, you're not going to significantly speed up the progress of the project as a whole. Yeah, so that is a hypothesis and I think there's a lot of truth to it. When we think about the ways in which AI can contribute, there are things we talked about before like the AI setting up their own curriculum and that's something that Ilya can't and doesn’t do directly. And there's a question of how much does that improve performance? There are these things where the AI helps to produce some code for tasks and it's beyond hello world at this point. The thing that I hear from AI researchers at leading labs is that on their core job where they're like most expert it's not helping them that much but then their job often does involve coding something that's out of their usual area of expertise or they want to research a question and it helps them there. That saves some of their time and frees them to do more of the bottlenecked work. And I think the idea of, is everything being dependent on Ilya? And is Ilya so much better than the hundreds of other employees? A lot of people who are contributing, they're doing a lot of tasks and you can have quite a lot of gain from automating some areas where you then do just an absolutely enormous amount of it relative to what you would have done before. Because things like designing the custom curriculum maybe some humans put some work into that but you're not going to employ billions of humans to produce it at scale and so it winds up being a larger share of the progress than it was before. You get some benefit from these sorts of things where there's like pieces of my job that now I can hand off to the AI and lets me focus more on the things that the AI still can't do. Later on you get to the point where yeah, the AI can do your job including the most difficult parts and maybe it has to do that in a different way. Maybe it spends a ton more time thinking about each step of a problem than you and that's the late end. The stronger these bottlenecks' effects are, the more the economic returns, the scientific returns and such are end-loaded towards getting full AGI. The weaker the bottlenecks are the more interim results will be really paying off. I probably disagree with you on how much the Ilya’s of organizations seem to matter. Just from the evidence alone, how many of the big breakthroughs in deep learning was that single individual responsible for, right? And how much of his time is he spending doing anything that Copilot is helping him on? I'm guessing most of it is just managing people and coming up with ideas and trying to understand systems and so on. And if the five or ten people who are like that at OpenAI or Anthropic or whatever, are basically the way in which algorithmic progress is happening. I know Copilot is not the thing you're talking about with like just 20% automation, but something like that. How much is that contributing to the core function of the research scientist?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.