Dwarkesh Patel: uncertainty

Transcript context

…That could be right. My sense is that some domains are structurally different in terms of how they operate and how much they depend on deep abstractions. Physics and math are much more on the side of being very far on the deep, hard-to-come-up-with-ideas side, whereas I think ML and most other domains are much more amenable to hill climbing. That’s my sense of how this will go in the future. Even in the regime where your AIs are having to plow — it’s 2030, a bunch of low-hanging fruit in research has already happened, and they need to make further progress — I still suspect that a bunch of the work will live more on the side of building increasingly complicated infrastructure and having really good intuition about what the experiments roughly look like. So I’m probably less sympathetic to the idea that the thing the AIs will lack is some deep insight. I’m more sympathetic to the idea that they really need a bunch of taste about in-the-weeds experiments that they currently don’t have. They need a bunch of intuition for what sorts of training approaches would work and what wouldn’t, in ways that current researchers have. Even in cases where there has been some breakthrough in AI, oftentimes in retrospect it looks like a big bottleneck to making that breakthrough happen was getting all of the micro details and mungy intuition right. An example of this is training AIs to be good at reasoning and chain of thought, doing RL on chain of thought. It looks like you probably could have done RL and chain of thought on GPT-3 and gotten kind of interesting results on math if you had really scaled it up and done a good job. But at the time, there was low-hanging fruit. Also, doing a good job with that training is kind of in the weeds on all the technical implementation and scaling it up and getting the hyperparameters right. So maybe you can demonstrate everything on Qwen 1B or whatever and get some sense that this whole thing is going to work. But people didn’t demonstrate it as early as they could have because of all of these other mungy details and intuition about exactly how to tune the parameters and how to set things up. This is my remaining skepticism, honestly, about this story. I’m not sure I understand why, if research breakthroughs are so amenable to intelligence, AI progress has not been historically faster than it could have been. As you were saying, by the time RLVR actually worked — even though you could have done it with less compute — we had to wait for oceans of compute, gigawatts of compute, to be available before people were doing this training, on the trajectory of compute continuing to increase so we make more breakthroughs. I don’t know. I feel like there were a lot of AI researchers in the year 2022 who were trying to crack reasoning. Was it just that they were bottlenecked by the ability to write infrastructure code, or what was happening? It’s a complicated mix. I think they would have gone faster if they could, as soon as they thought of an experiment, run that experiment without bugs, without bugs being very important. And then another part of it is that being able to run a lot of experiments at high compute lets you paper over ways in which the way you implemented it isn’t quite right or you didn’t have the right hyperparameters. So compute is just really helpful for doing AI research, and you can cover over a lot of things. But that doesn’t mean that massive increases in labor wouldn’t also be helpful, especially if that labor comes with among the best intuitions that people have in the field. I just think that’s really helpful. Another part of my perspective here, which is maybe a bit different from where you’re coming from, is that I’m expecting somewhat more transfer than you seem to be imagining. I’m imagining these AIs are actually pretty good scientists in general and are pretty reasonable at all of that stuff. When you interact with them, it’s not like they have some really hyper-specialized savant-type vibe. They’re actually pretty good at all of the stuff in R&D, and then maybe extremely good at some subdomains. So they’re incredibly superhuman at writing kernels, incredibly superhuman at everything with very short feedback loops, and then pretty good at all the other stuff, totally able to match other people. I think we are seeing this now. When I look at AIs right now, it’s already the case that they can pretty competently match humans who are mediocre at ML research at doing ML research. It’s just that being mediocre at ML research is not that helpful. The thing you actually want are people who are good at ML research. My sense is the AIs are just improving at all of these things. Their taste is improving, their intuition is improving, and it’s already the case that their taste and intuition is not complete garbage.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.