Nathan Labenz: belief

Transcript context

…much a process of natural selection, of evolution, as of learning. So, for example, we presumably in our brains have a mechanism for representing the meaning of words. We still don't know what it is. Some people, by the way, psychologists, had long posited a mechanism that would be quite analogous to what we now understand goes on in large language models. Uh, but in any event, I think it's pretty safe to say that that is a product of natural selection, right? Now, also in the course of the training, the machine becomes conversant in a specific human language. Well, that's more a product of human learning, right? During, uh, you know, as, as the organism is developing. But I think what laypeople need to understand, and I'm, again, I'm, I'm curious as to your view on, on, as to how this would hold up in AI circles, but is that these things basically, in a certain vague sense, re- recapitulate evolution. Recapitulates, in a way, a misleading word, because the cognitive functionality is not the exact same mechanisms the brain uses, but, you know, I think close enough in a lot of, uh, cases. It's kind of doing millions and millions of years of evolution in a few months, and you can, you can do a lot with that, and we have not begun to, I think, exhaust the potential of that. I mean, as so far as words go, we're close maybe, but There's a lot more to do. Do you, do you think people in the field would agree that, yeah, it's, it's a lot like i- i- it's really, evolution is maybe a better term for many purposes than learning? Yeah, it's a great question. I think there's a couple different senses, and I'm not sure I have gripped all the senses that you mean when you use that term. You certainly do hear people talk about pre-training broadly as sort of being analogous to evolution in the sense that there's y- there's sort of this question of like, well, why are humans so sample efficient, right? And the models need so long to train. Mm-hmm.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.