Keith Duggar: evaluation

Transcript context

…ing to threshold this and make it boolean again, you will see what is the concept that that learned, or you can leave it in in in in numeric form. So the learning is actually extraordinarily powerful. I've always thought, and you know, I think, you know, a lot of people in deep learning really believe this, that, you know gradient descent can do amazing things provided you give it the right architecture to operate on. And in a way what all these million papers are about is about finding the right architecture for gradient descent to operate on, and of course transformers are a great leap forward, but I think transfer I think, you know, Tensor Logic is an even greater leap forward. How so? Because, for example, I can picture suppose we we we wanna get rid of Python. So, like, I'm over here in, you know, PyTorch, and and I've described all my my layers kind of in the clunky syntax. And instead, I'm like, no. Now I have, you know, the Tensor Logic, you know, GitHub programming language. Let me go do there. I'm still going to construct my layers. Right? So because, like, for example, you do of course, you allow for the the, you know, nonlinearities. Right? So after every EIN sum, I can apply whatever kind of nonlinear function I want to ReLU or sigmoid or whatever else. Right? That's still gonna be described in my program. It's like I'm gonna have this shape EIN sum followed by this linearity feeds into this shape followed by so I'm still gonna have to do that kind of, like, you know, structuring of the network, if you will, except now in Tensor Logic. And in my opinion, that's 1 of the biggest limitations right now is these are all just divined cantation structures that people have come up with. Like, let's put in a dropout layer here and this kind of layer there and there. We don't actually allow the machines to learn the overall topological structure. We only allow them to to find weights within that structure. No. But okay. I understand your question. But tensor logic does allow that. You know, step 1, you can you can encode a multilayer perceptron, the entire multilayer perceptron, and I do that in the paper with a single tensor equation. All the layers provided that they all use the same nonlinearity can be encoded in 1 equation. Okay? Number 1. You can also have different equations for different layers or typically sets of layers, you know, to your However way you please. But from the point of view of structure discovery, the thing to realize is that if you create, if you set up 1 of these very general equations that you can in tensor logic, that in some sense it can, it's a very broad classes of architecture, then what the learning does is it discovers the architecture within that space. Right? Which if you think about it at some level is what a neural network, when you compare an ordinary multilayer perceptron with a set of rules, right? A multilayer perceptron is you you can take in fact there was a system called k band in the early days that did this, very clever. It initialized a a a a multilayer perceptron with a set of rules because each neuron is a rule. Right? But it's also more flexible because now you can have weights. Right, but but if but you know, neuron, a single neuron can represent a conjunction and therefore a layer can represent a junction and so forth. So when you're learning weights in an ordinary neural network, you can actually see it as learning the structure of a set of rules. What tensor logic is doing, this is at a more powerful level, like like that was just propositional and now this this is at the full level of generality of of first order logic. But you can learn the structure and then of course then there's more than 1 way to do that, and you can also decide how black and white you want the structure to be, what you want to leave as weights, what you want to discretize. But the structure itself can be learned by taking a tensor equation. A tensor equation is a very general thing, right? When when you learn the weights of those tensors that the that materializes to a specific network structure.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.