Ryan Greenblatt: belief

Transcript context

…No one person would’ve known enough about topology and algebraic whatever in order to make some counterexample to a big conjecture. My view is that ML is a less deep domain than math, and so there’s less of a thing where there are individual experts with really deep expertise in some area that they combine, but there’s definitely going to be some of that. But then I also think that ML has some attributes that make it even more favorable to AI training than mathematics in some ways. In particular, you can get a better sense of whether you’re succeeding, and you can see intermediate progress. In math, it’s often the case that there’s no easy way to see whether or not you’re close to success. Whereas if your goal is, for example, to get to some training loss 2x faster, you can kind of see when you’re halfway there. It tends to be the case that ML innovations are very additive, or maybe multiplicative depending on how you think about it, where basically you can keep stacking innovations. Usually the innovations just add together and don’t interfere with each other, though obviously it’s going to depend on the details. So I think that in a lot of ways, AI R&D will have properties quite similar to math, where you can train on chunks of AI R&D that are pretty similar in structure to the problem you actually cared about, in a very verifiable way, and then that will transfer. There’s an open question of exactly how well it will transfer, but I think that the transfer currently for math looks pretty good. My expectation is that the transfer for AI R&D will look pretty good, but not amazing. So one concern I have is that I think even in mathematics, as far as I’m aware, we have not seen very impressive new theory. We’ve seen a lot of impressive, verifiable, specific results — for example, find a counterexample to this conjecture — but we have not seen “come up with the idea of topology“ kinds of levels of things, or “come up with things like group theory“. It seems like ML research has elements of both of these things. But the less verifiable thing of coming up with new ways of thinking about the problem would be harder to induce. Take, for example, the idea of scaling laws. Obviously, there is some end verification loop such that you can train GPT-4 better if you have the idea of scaling laws from 2020. But there is a longer and potentially more compute-laden road to inducing AIs to be like, “Okay, I got to think carefully about how I should be scaling my parameters and data. What are different kinds of investigations I could run to understand this? Maybe I can come up with a visualization and an isoFLOP analysis or something.” But that does seem like a longer verification loop than just, “Hey, let’s get nanoGPT loss to go down.”…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.