Pedro Domingos: belief

Transcript context

…You've described a language and certainly if components of that language are Turing complete, that's a big vexed issue, we'll come to that a little bit later. But because of computational equivalence, we can, you know, from an expressibility point of view, we can describe anything in the universe. So we've got this framework. But to me, the challenge in AI is structure learning. Right? So as well as being able to express stuff, it's being able to adapt to novelty and and create perhaps from building blocks that we already have a new structure to allow us to do something useful in that domain. And I can't quite make that leap with with your technology yet. So how how do we do the meta thing where we actually build the the tensor logic constructions to represent the kind of world that we're seeing? Oh very good. So I actually go into that in the paper, know, but briefly the paper is just you know, an informal introduction to these ideas. Inductive logic programming, right, is the field that deals with discovering rules from data. But it does this by things like greedy search or beam search and it's a very large you know search space and it's extremely inefficient, right? Which is actually 1 of the things that killed it even though it could do all these things that people know in deep learning are just painfully rediscovering. In tensor logic, this is 1 of the best parts of it, the structural learning falls out of the gradient descent. The gradient descent actually does structural learning. And then on top of that, this is actually the the the best part as as far as the learning is concerned, there is this thing called predicate invention, which is discovering new predicates, discovering new relations that are not in the data but that explain it better. I would say that you know in some sense discovering representation like that is the key problem in AI, is the holy grail. Everything that we know, you know, like when you look at it at the world, right, you don't see pixels. You don't see photons hitting your retina, right, you see objects. The objects are invented predicates. All the way up to in science, right, the most like Newton's genius was to introduce a new a new quantity which is force, and energy, and entropy, and all these etcetera etcetera. Right? So in in tensor logic that also just happens by gradient descent. Right? It's it's, you know, it's hard to believe, but let me just give you a hint as as as to why this is the case. There's this other thing that is folded into tensor logic which is tensor decompositions. Right? And tensor decompositions are generalizations of matrix decompositions. And if you think about matrix decompositions to take that simple case, what a matrix decomposition does is it takes a matrix and decomposes into 2 new matrices that together are more compact but essentially reproduce the same data, right? And there's a generalization of that 2 tensors called the Tucker decomposition, there's others but the Tucker 1 is is the most relevant 1 here. And so if you write in tensor, to answer your questions very directly, if you write in tensor logic a rule schema, you know, including you know a a data tensor on the left hand side, and by the way your entire data can just be reduced to 1 tensor embedded as 1 tensor, we can touch on that later. But you write a rule expressing that as a function of a few other tensors, and the gradient descent, just as in matrix factorization, will discover the best values for those. And then if you want to, for example, then discretize it, say like I'm going to threshold this and make it boolean again, you will see what is the concept that that learned, or you can leave it in in in in numeric form. So the learning is actually extraordinarily powerful. ing to threshold this and make it boolean again, you will see what is the concept that that learned, or you can leave it in in in in numeric form. So the learning is actually extraordinarily powerful. I've always thought, and you know, I think, you know, a lot of people in deep learning really believe this, that, you know gradient descent can do amazing things provided you give it the right architecture to operate on. And in a way what all these million papers are about is about finding the right architecture for gradient descent to operate on, and of course transformers are a great leap forward, but I think transfer I think, you know, Tensor Logic is an even greater leap forward.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.