Pedro Domingos: evaluation

Transcript context

…Yeah. So I I I understand that. Let me bring this back to like to the folks who are familiar with, you know, PyTorch or traditional techniques. What you described is, yeah, I can just create a fully connected, you know, network with however many layers I want and then let SGD, you know, find all the weights. That doesn't work. Like, it doesn't work in practice, and it's not gonna work with tensor logic. You know, it's just a different representation of the same fundamental problem, which is there's too many degrees of freedom. It's not gonna learn anything useful. This is why so much alchemy goes into structuring, you know, constrained networks to have certain, you know, built in, you know, inductive bias. Right? No. Absolutely. So to take another example, you can also do an entire ConvNet in just 1 tensor equation. And and, you know, like, the quintessential example of, like, yes, complete connections don't work is a multilayer perceptron for for vision, right, which you replace with a convnet that actually has a local structure. That is also a tensor logic equation. Now we are saying, how do you choose between the convnet and then MLP, right? Very good question. And now there's a range of things you can do. You can actually these days start out with a very general structure because, I mean, GPUs and and and large, you know, server farms are an amazing amount of power for something like this. Right? So you can almost, I would say, brute force that search provided you have the data. I'm not actually recommending you to do that, right? You can also, however, and more interestingly, you can, and this is actually 1 of the key benefits of tensor logic is that you can write down what you believe are properties of the say, right now what happens in in in in, know, when you for example program a network in Python is like, you have to commit, you say like, here's the structure, and now the, you know, the the learning of the only thing that happens is the learning of the weights. Intensive logic you don't have to do that. You can you can just you can set up 1 of these very general structures and then you say, let me give you a bunch of equations that are things that I believe to be true about the structure but do not completely determine it. And those just work like priors, and indeed like soft priors, right? Those can and then again you can turn up the temperature on this or down, and say like, you got to obey this equation, and that 1, you know, sure you can overwrite. And then and then this, in my experience, this is actually what is important, is that the gradient descent, instead of starting from a tabular rasa, have this kind of soft knowledge. And then most importantly like you, the developer, you the AI researcher, you you get to this is really the essence is that it's not this you every every, you know, deep learning researcher or data scientist know this. It's like you don't do the same priority and then push the button and hope for the best, right? It's like there's an iterative loop of you set up the structure and then you learn, you get the results, and then you refine the structure. And what this does is it makes that more efficient much more because you just have to in your interpreter you write 1 more equation or you modify an existing equation, and also the entire stack is is of what you learn is much more interpretable than it was before. It's actually in some ways 1 of the most important properties of tensor logic, is that you can understand what's going on much better than you could in 2 ways. h more interpretable than it was before. It's actually in some ways 1 of the most important properties of tensor logic, is that you can understand what's going on much better than you could in 2 ways. 1 is that the code is much more transparent than the whole pile of things that you have sitting under a bunch of, you know, PyTorch procedure calls, but also the result of learning, at least if you do it in in certain ways that I discuss in the paper, the result of learning is transparent in the way that a transformer just you know can hope to be.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.