Dwarkesh Patel: belief

Transcript context

…Our scenario focuses on coding in particular because we think coding is what starts the intelligence explosion. So we are less interested in questions of like, “how do you mop up the last few things that are uniquely human” compared to “when can you start coding in a way that helps the human AI researchers speed up their AI research, and then, if you’ve helped them speed up the AI research enough, is that enough to, with some ridiculous speed multiplier- 10 times, 100 times- mop up all of these other things?” One observation I have is, you could have told a story in 2021, once ChatGPT comes out… I think I had friends who were credible AI thinkers who were like, “look, you’ve got the coding agent now, it’s been cracked. Now the GPT4 will go around and it’ll do all this engineering and we do this RL on top. We can totally scale up the system 100x” and every single layer of this has been much harder than the strongest optimist expected. It seems like there have been significant difficulties in increasing the pre-training size, at least from rumors about field training runs or underwhelming training runs at labs. It seems like building up these RL- total outside view, I know nothing about the actual engineering involved here- but just from an outside view it seems like building up the O1 RL clearly took at least two years after GPT4 was released. And these things are also, their economic impact and the kinds of things you would immediately expect based on benchmarks for them to be especially capable at isn’t overwhelming, like the call center workers haven’t been fired yet. So why not just say look, at higher scale it will probably get even more difficult. Wait a second, I’m a little confused to hear you say that, because when I have seen people predicting AI milestones like Katja Grace’s expert surveys, they have almost always been too pessimistic from a point of view of how fast AI will advance. So I think the 2022 survey, they actually said that things that had already happened would take like 10 years to happen, but then the survey- it might have been 2023, it was like six months before GPT3, GPT4, came out. And there were things that GPT3 or 4 or whichever one of them it was, did, that it did in six months that they were still predicting like five or ten years from. I’m sure Daniel is going to have a more detailed answer, but I absolutely reject the premise that everybody has always been too optimistic.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.