Tyler Cowen: prediction

Transcript context

…rich records of advanced creative activity — the cave paintings, for example, which are pretty remarkable. I was lucky enough to be able to get into Lascaux before it was sealed, and it’s astonishing. You can understand why Picasso, when he saw it, said, “We haven’t learned anything in tens of thousands of years.” Well, Lascaux is fairly recent, maybe 30,000 years ago, but no, goes back to maybe 80,000 years. These seem like long numbers, but in evolutionary time, these are the flick of an eye. Well, what does all this suggest? Suggests that something happened along with the appearance of modern humans, namely the emergence of these capacities that we’re talking about, that amazed Galileo, Humboldt, and others. And nothing’s changed since. There’s been no change that we can detect in the nature of these cognitive capacities, which seem to be species properties of humans in the technical sense, meaning common to all humans (apart from extreme pathology) and completely unique — nothing like them anywhere in the animal world. Will misunderstandings of Darwinian evolution lead you to think that this is inconsistent with it? It’s actually quite consistent with the modern theory of evolution. Darwin did have a famous statement saying that all evolutionary changes have to be very small. He even said, “If this turns out not to be true, my total theory collapses.” Well, it’s now known that that’s just not true. There are major changes that take place. Suddenly, a lot is known about this. One of them may well have been, seems to have been whatever happened pretty much along with the appearance of modern humans which provided these capacities, which again have no analog in the animal world and are common to the species. Now, you’ve been very critical of large language models. There’s a recent essay by Stephen Wolfram, where he argues the success of those models is actually evidence for your theory of language — that they must, in some way, be picking up or detecting an underlying structure to language because their means are otherwise too limited to be successful. What’s your response to that view? He’s a brilliant scientist. I’ve talked to him sometimes. I think this is partly true but partly misleading. The large language models have a fundamental property which demonstrates that they cannot tell you anything about language and thought. Very simple property: its built-in principle can’t be modified, namely, they work just as well for impossible languages as for possible languages. It’s as if somebody came along with a new periodic table of the elements which included all the elements and all impossible elements and couldn’t make any distinction on them. It would tell us nothing about chemistry. That’s what large language models are. You give them a data set that violates all the principles of language, it will do fine, doesn’t make any distinction. What the systems do, basically, is scan an astronomical amount of data, find statistical regularities, string things together. And using these regularities, they can make a pretty good prediction about what word is likely to come next after a sequence of words. A lot of very clever programming, a lot of massive computer power, and of course, unbelievable amounts of data, but as I say, it does exactly as well with impossible systems as with languages. Therefore, in principle, it’s telling you nothing about language. Like all organisms, we have innate capacities. There’s something about our genetic endowment that determines that the embryo grows arms not wings, and there’s something about the genetic endowment that says that a newborn infant can instantly pick out parts of the noise that surrounds it and say to itself, “Those parts are language. I’m going to” — not consciously, of course; it’s all totally reflexive — “These parts are language. I’m going to pursue a course of maturation, a well-determined course of maturation, which means that by about two or three years old, I’ve basically absorbed the fundamentals of language.” Now, you can take the smartest chimpanzee or the dogs under my desk — they can listen to this noise forever. They have no idea there’s anything there but noise. Well, that’s a fundamental property of humans built in. It’s the reason why you and I can be having this discussion now, but a troop of chimpanzees can’t be.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.