Dario Amodei: uncertainty

Transcript context

…We talked three years ago. In your view, what has been the biggest update over the last three years? What has been the biggest difference between what it felt like then versus now? Broadly speaking, the exponential of the underlying technology has gone about as I expected it to go. There’s plus or minus a year or two here and there. I don’t know that I would’ve predicted the specific direction of code. But when I look at the exponential, it is roughly what I expected in terms of the march of the models from smart high school student to smart college student to beginning to do PhD and professional stuff, and in the case of code reaching beyond that. The frontier is a little bit uneven, but it’s roughly what I expected. What has been the most surprising thing is the lack of public recognition of how close we are to the end of the exponential. To me, it is absolutely wild that you have people — within the bubble and outside the bubble — talking about the same tired, old hot-button political issues, when we are near the end of the exponential. I want to understand what that exponential looks like right now. The first question I asked you when we recorded three years ago was, “what’s up with scaling and why does it work?” I have a similar question now, but it feels more complicated. At least from the public’s point of view, three years ago there were well-known public trends across many orders of magnitude of compute where you could see how the loss improves. Now we have RL scaling and there’s no publicly known scaling law for it. It’s not even clear what the story is. Is this supposed to be teaching the model skills? Is it supposed to be teaching meta-learning? What is the scaling hypothesis at this point?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.