Pedro Domingos: prediction

Transcript context

…So we've covered some interesting topics on MLST before, I mean of course there's geometric deep learning, which is this idea that symmetries are are fundamental. We've spoken with Andrew Wilson from NYU recently about soft inductive priors, and I've just spoken with Yee Ma about his crate series of architectures. And I guess the the prevalent idea here is almost platonistic, that there are real natural patterns. And if we kind of bias the model as you were just alluding to, that it will converge on really good representations that describe reality. Now the the alternative view is that reality is is constructive and gnarly and and that won't work. But you were talking about your Tucker decomposition earlier and that's this idea that, you know, we might have a large sparse matrix, we might want to densify it, we might want to factorize it and the factorization will kind of pull out some of these natural orderings, you know, of the universe perhaps. And I guess I was thinking, isn't it a bit like a GZIP algorithm? I mean, what if these factorizations are just semantically meaningless? How do you know that you've got a good 1? You know, great question and you've touched on several things there. Let me start with, you know, the geometric deep learning, right? I'm a big fan of this. In fact, you know, I gave a keynote at the second iClear on something that I called symmetry based learning, which is in some ways an ancestor of geometric deep learning. I really do think that the universe possesses these fundamental symmetry Actually I don't think that. This is known, right? In physics, right, the standard model is basically a bunch of symmetries. And this is extraordinarily powerful, right, that such simple things could be such universal regulators that you then basically can build everything else out of, right. And and if you think about it, in machine learning the problem is like what is the learning bias that you should start from, Right? Should you pull in a lot of knowledge? Should you have a very, you know, very vague architecture? The thing about machine and there's the no free lunch theorem, right, that says you know, if you don't assume anything you can't ever learn anything. The thing that's amazing about machine learning is that with very weak biases you can get very far. Right? And I would submit that those weak biases fundamentally at the end of the day the most important ones are these symmetries. And tensor logic is precisely, you know, I think the perfect language for expressing those symmetries as the physicists will tell you, right? It's what it's what they use in in like, in not the logical, you know, version but but the numeric version. Right? So so I I think we can discover those regularities. I have some suspicions as to what they might be, but I think, you know, we're not quite there yet. But I think once we have those regulators in some sense, you know, a a they will play in AI the role that the standard model plays in physics. Right? Now of course, as you say, you know, there are people who say like, oh, forget that. Right? You know, going back to Marvin Minsky, right, there's like, there is no small set of AI laws or anything. It's just 1 damn thing after another, blah blah blah blah. Right? Like you're dreaming. Right? And I respect that point of view. Right? And and you know, we will find out empirically. But what I if I had to guess how this is gonna play out, at the end of the day it's gonna be like this. This stuff that I'm talking about gives you, you know, the 80 20. You know, it gets you 80% of the way. And then the other 20% away, you have to do a lot of these things, you have to do a lot of hacks, etcetera etcetera. But something like sense Tensor Logic still makes it much easier and faster to do those hacks than than if you didn't have it. So you'd actually gives you benefit both in the 80% part and in the 20% part. There are folks, you know complexity science, there's this guy called David Krakauer and in his book on the first page actually, the very first sentence, the scientific and social implications of differences between a closed reversible symmetry dominated and predictable classical domains. I think that's what you're talking about, the kind of the Roger Penrose type world. And b open self organizing dissipative uncertain and adaptive domains. Now I think the latter is where all the interesting stuff in the universe is. It's where life and intelligence and all the all the stuff we want to model. And could it be the case that those things are not reducible in the way that you're arguing they are?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.