Scott Alexander: prediction

Transcript context

…My guess is that by the end of this year there’ll be something that can kind of do that, but unreliably. And that if you actually tried to use that to run your life, it would make some hilarious mistakes that would appear on Twitter and go viral, but that the MVP of it will probably exist by this year. Like there’ll be some Twitter thread about someone being like, “I plugged in this agent to like run my party and it worked!” Our scenario focuses on coding in particular because we think coding is what starts the intelligence explosion. So we are less interested in questions of like, “how do you mop up the last few things that are uniquely human” compared to “when can you start coding in a way that helps the human AI researchers speed up their AI research, and then, if you’ve helped them speed up the AI research enough, is that enough to, with some ridiculous speed multiplier- 10 times, 100 times- mop up all of these other things?” One observation I have is, you could have told a story in 2021, once ChatGPT comes out… I think I had friends who were credible AI thinkers who were like, “look, you’ve got the coding agent now, it’s been cracked. Now the GPT4 will go around and it’ll do all this engineering and we do this RL on top. We can totally scale up the system 100x” and every single layer of this has been much harder than the strongest optimist expected. It seems like there have been significant difficulties in increasing the pre-training size, at least from rumors about field training runs or underwhelming training runs at labs. It seems like building up these RL- total outside view, I know nothing about the actual engineering involved here- but just from an outside view it seems like building up the O1 RL clearly took at least two years after GPT4 was released. And these things are also, their economic impact and the kinds of things you would immediately expect based on benchmarks for them to be especially capable at isn’t overwhelming, like the call center workers haven’t been fired yet. So why not just say look, at higher scale it will probably get even more difficult.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.