Tim Scarfe: prediction

Transcript context

…artificial or natural, or whatever adjectives you add to intelligence, we have to be very specific. It's a very loaded world, right? I mean, even intelligence itself may have different levels, different stages, right? So it's high time we clarify that concept scientifically and mathematically, right? So that we'd be able to talk about study intelligence, the mechanism behind it at each level. There are some more unified principle behind, even different stages of intelligence. There's something in common. There are also things that are different. So it's high time we do that. 1 of the intelligence at the level is that common to animals and human, right? Human, we are animals. That level of intelligence is what we think are very common to all life, which is how memory, how we learn knowledge about the external world, and then memorize as part of our memory, and use that to predict, to react to the world, help us make decisions, predict and make better decisions for survival and so on and forth. That's very, very common. And this is the level of intelligence we're talking about very much for the book as well, right? And hence, for this level of intelligence, how our memory works, today we also have a fancy word for memory. We call it a world model, right? And it's how we develop such a memory, such a world model, and how the model gets evolved, and how we use it. That's actually, this is the level of we talk. So we actually believe that for this level of intelligence, for how our memory formed and how they work, is precisely the 2 principles are incredibly important. And we believe they are necessary, is that memory or knowledge is precisely trying to discover what's predictable about the world. Hence, for all understand, all such information are have intrinsically very low degree of freedom. We call it low dimensional structures. And hence, the way to pursue such knowledge is precisely through trying to find the most simple representation of the data. And hence, compression, denoising, dimension reduction is actually all just different words to pursue such knowledge, such a structure. And hence, that's the word captured by the word parsimony. Finding, you know, explaining, making things as simple as possible, but not any simpler, right? So this is Einstein says this word, this is the sentence Einstein used to describe science. Actually, that is also what the intelligence, at least at this level, is precisely doing the same thing. The second part of the sentence, not any simpler, precisely says, consistent, consistency. Make sure your memory is actually consistent with be able to recreate, simulate the world just right, not any similar. If you're simpler, you may lose part of the predictability, and also ability to predict it well. And so that's actually the those 2 actually coexist, we believe. At least those are the 2 principles, parsimony and the consistency or self consistency, are actually the 2 characteristic about how our memory works. So we want to have understanding which carves the world up by the joints, which represents the important invariances in the world. And the thesis is, I think, that compression might be necessary for understanding. My possible concern with that is that what we are doing with machine learning is representing extant examples of a long phylogenetic tree of evolution. Mhmm. So to what extent does knowing their representation now help us? Do we also need to know how they evolved and where they might go in the future? The the process to acquire knowledge, to gain information about our side world, that's a compression. Find what is compressible. What has orders? What phenomena has orders? Has low dimensional structures that allow us to predict, to rule out variabilities, to predict world tomorrows, or predict world better, in essence. So in a sense that that is the ability we believe is really what intelligence is all about, at least the common intelligence we're talking about, right? We can talk about the more higher level intelligence later. And if you look at the history of life, how life was developed, So we actually come to believe, right? You know the mechanism that we laws that governs the physical world, we call it physics, right? But what is the mechanism that governs the evolution of life? I think it's intelligence, right? Even the process you mentioned that through the evolution and life evolves, precisely they learn more and more knowledge about the world, and they encode them through DNAs to pass it on to next generation. And that is a compressing, that's a process to compress knowledge that learn about the world through our DNAs. But the mechanism to update it is actually very brutal, very brute force. And through, you know, random mutation and the natural selection. Yes, it does evolve. It does advance, but at a huge cost of resource, time, and also very unpredictable. Which if you're acute, you probably observe there's some similarity with how current big model evolves, right? Many, many groups try without principle, trial and error, empirical. And the lucky ones survive and gets advocated everywhere and become very, very popular, right? Dominate the practice. So in a sense that it can make it an allergy, right? I think to the people, students ask me at which stage our artificial intelligence is at today. Then there's already an allergy in nature, right? We are very much at the early stage of the life form, right? And so hence, that is a compression process. That's a process that also gain knowledge about the world. But of course, on, we develop individual animals to develop the brain, develop neural systems, develop senses, including visual and touch and so on. So we actually start to use a very different mechanism to learn, to compress our observations, to learn knowledge, and to build memories of Oxford world. And even individuals start to have that ability, rather than just inherit knowledge from their DNAs. So that's a different stage. Then that part of the knowledge is no longer encoded in our genetics, in our genes, but also in our brains. And that's actually a level of intelligence we talk about most the time these days, you know, which is common to animals, which is common to humans, and the knowledge or the intelligence we talk about what brain functions.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.