Dwarkesh Patel: uncertainty

Transcript context

…It’s relevant also not just in terms of verified reward, but presumably, the end goal is understanding, human understanding. Even if you do have some thousand-page proof of some math thing or some grand new physical theory, the goal is understanding. Maybe if the goal is predictiveness, you can just have automated engineers go off and build rocket ships where we have no idea how they work, but we can get between stars. But there are going to be a lot of people who want to understand. You’re still going to want whatever the concision function is that distills down this complicated way of thinking into the right one, like the equivalent of the universal law of gravitation for Newton. You would still want to train AIs to be able to do that and find the compressed representation. People have this worry about mathematics in particular that AIs will prove the Riemann hypothesis, and our understanding of mathematics won’t be any the better for it. I have a couple of questions about this. The first one is whether this is something you should expect. Isn’t the reason humans come up with general, natural objects and subgoals when we’re working on a big problem that this is just useful when you’re trying to work on a complicated, important problem? Theoretically, would this even be a simpler way to solve the Riemann hypothesis, as opposed to just coming up with the natural abstractions that are relevant to thinking about the problem? And then two, empirically, is this what we observe when AIs make progress on problems today? When the AI came up with that counterexample to the unit distance conjecture, you can just read its chain of thought. It’s not understandable to me, because I don’t know anything about mathematics, but it seems that to other mathematicians it was understandable. It made use of known concepts of mathematics and proved relationships between them, all in natural language. As a result, it accelerated our understanding of the connection between this object and this conjecture. Empirically, is this a thing we should be worried about? I think it depends on the nature… If we break down the three possible ways of solving the Riemann hypothesis… The other big one from this year was a certain Erdős problem numbered 1196, about these things called primitive sets. It had that character of bringing an idea from a seemingly different field. As soon as you present the basic idea to a mathematician… You say, “What if we try the Markov chain process where we show that this thing is one from the bottom up probabilistically rather than the top down, and use the von Mangoldt function?” If you say that to someone in the know, they’d know how to run with it. You have this very small idea that has the form of expertise in one field and expertise in another, drawing a little lightning bolt between them. Those are going to be very human-parsable, because all you have to do is show the start and end point of what those connections are. If the character of it is mountain building, you have to put in a lot more time to understand that new mountain that was built, because it’s a new thread, not just a lightning bolt between them. And if the nature of the progress was just raw hustle—a super long chain of reasoning with no new theories—then you would have that worry of this whole digestion process. So I don’t think there’s one clear answer. It depends on what the solution would look like. On the mountain building side, that would actually be really interesting to see. Is it by default very human-understandable, the way we see new theories from great mathematicians? Or is it an alien, different kind of mountain being built where we have to reprocess the kinds of abstractions we engage with? The closest example here would be the attempted solution of the abc conjecture. We maybe shouldn’t get into that one, but it probably is not a correct solution. Basically it’s this whole new way of thinking that this otherwise reputable mathematician in Japan had come up with. It took mathematicians a long time to even parse what he was saying, but it had the feeling of an alien bit of mathematics that’s theory building, not just a long chain of reasoning. He called it inter-universal geometry. The biggest fear would be that an AI does that, and then much like the abc conjecture, people work for years to go up the mountain, and they’re like, “Dang it. This just isn’t right.” If it turns out to be wrong, but it really looked right. Even if it was right, there’s just a lot of effort to hike up a new mountain.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.