Terence Tao: evaluation

Transcript context

…Often in the history of science when a new theory comes up that in retrospect we realize is correct, it seems to make implications that either make no sense because they’re wrong, and we realize later on why they’re wrong, or they’re correct but seem wildly implausible at the time. As you talked about, Aristarchus had heliocentrism in the third century BC. The ancient Athenians were like, “This can’t be because if the earth is going around the sun, we should see the relative position of the stars change as we’re going around the sun, and the only way that wouldn’t be the case is if they’re so far away that you don’t notice any parallax,” which is actually the correct implication. But there’s times when the implication is incorrect and we just need to graduate to a better level of understanding. Leibniz would chide Newton and disagree with Newton’s theory of gravity on the basis that it implied action at a distance, and they didn’t know the mechanism, and Newton himself was sort of stunned that inertial mass and gravitational mass were the same quantity. All these things later were resolved by Einstein. But it was still progress. So the question for a system of peer review for AI would be: even if you can falsify a theory, how would you notice that it still constitutes progress relative to the thing before? Often, the ultimately correct theory initially is worse in many ways. Copernicus’s theory of the planets was less accurate than Ptolemy’s theory. Geocentrism had been developed for a millennium by that point, and they had made many tweaks and increasingly complicated ad hoc fixes to make it more and more accurate. Copernicus’s theory was a lot simpler but much less accurate. It was only Kepler that made it more accurate than Ptolemy’s theory. Science is always a work in progress. When you only get part of the solution, it looks worse than a theory which is incorrect but somehow has been completed to the point where it kind of answers all the questions. As you say, Newton’s theory had big mysteries. They had the equivalence of mass and action at a distance, which were only resolved with a very conceptually different approach centuries afterwards. Often progress has to be made not by adding more theories, but by deleting some assumptions that you have in your mind. One reason why geocentrism held on for so long is we had this idea that objects naturally want to stay at rest. This is the Aristotelian notion of physics, and so the idea that the Earth was moving… How come we weren’t all falling over? Once you have Newton’s laws of motion—an object in motion remains in motion and so forth—then it makes sense. Conceptually, it’s a very big leap to realize that the Earth is in motion. It doesn’t feel like it’s in motion. The biggest advances, like Darwin’s theory of evolution, is the idea that species are not static. This is not obvious because you don’t see evolution in your lifetime. Well, now we actually can, but it seems permanent and static. Right now we’re going through a cognitive version of the Copernican revolution, where we used to think that human intelligence is the center of the universe, and now we’re seeing that there are very different types of intelligence out there with very different strengths and weaknesses. Our assessment of which tasks require intelligence, which ones don’t, has to be reordered quite a bit. Trying to fit AI into our theories of scientific progress and what is hard and what is easy, we’re struggling quite a lot. We have to ask questions that we’ve never really had to ask before. Or maybe the philosophers had, but now we all have to deal with it. This brings up a topic I’ve been very curious about. You mentioned Darwin’s theory of evolution. There’s this book, The Clockwork Universe by Edward Dolnick, which covers a lot of this era of history we’re talking about. He has this interesting observation in there. The Origin of Species was published in 1859. Principia Mathematica was published in 1687. So The Origin of Species comes out two centuries after Principia. Conceptually, it seems like Darwin’s theory is simpler. There’s a contemporaneous biologist to Darwin, Thomas Huxley, who reads The Origin of Species and he says, “How stupid not to have thought of that.” Nobody ever says that about Principia, chiding themselves for not having beaten Newton to gravity. So there’s a question of why did it take longer? It seems like a big part of the reason is what you were saying. The evidence for natural selection is overwhelming in a certain sense, but it’s cumulative and retrospective, whereas Newton can just say, “Here are my equations. Let me see the moon’s orbital period and its distance, and if it lines up, then we’ve made progress.” Lucretius actually had this idea that species adapted to their environment in the first century BC but nobody really talks about it until Darwin because Lucretius couldn’t run some experiment and force people to pay attention. I wonder if we’ll in retrospect end up seeing much more progress in domains which have this kind of tight data loop where you can verify them quite easily, even though they’re conceptually much more difficult.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.