Andrew Dudzik: evaluation

Transcript context

…So here's the key connection to programming. In functional languages, we define data types like lists recursively. A list is either empty or it's an element followed by another list. Categorically, this is an algebra for an endofunctor. The structure map of the algebra packages together all of the constructors of the data type. And the homomorphism from this algebra is exactly what programmers call a fold A function that consumes the list by recursively applying some operation. So the framework is describing the very structure of recursive computation. When you write code, you you encounter syntax errors. You don't so much encounter semantics errors. So the syntax is really quite grounded in the things that you're actually typing in when you're when you're writing something, whether it's an ordinary algorithm, whether it's a network architecture, etcetera. The semantics is much more about like how can programs behave. And, you know, 1 example of this for, we have something like list types. So lists are defined by a type constructor. Given a type t, you have another type, list of t. So what are the semantics of lists? Well, the semantics of lists are really things that are foldable, sort of foldable types. So, like, numbers with addition, they're a sort of foldable type. If I have a list of numbers, I can just add them to sort of remove the list. And now these these foldable types, mathematicians have a very different name for them, monoids, which are a more general kind of group. But in any case, like, that's sort of the the semantics of lists. And, before I say something about syntax, let me say that our paper is really mostly exploring things from the semantic angle. Why is this? It's because 2 different syntaxes can describe the same thing, very easily. You know, we could have an arithmetic theory where we have addition and we also have negation, or we could instead have subtraction. And you can describe the same things in those 2 different languages, but the languages really are different. They give you the same semantics, but the syntaxes are different. And so when doing mathematical analysis, proving theorems, it's often really beneficial to work, from a semantic point of view. But it's worth really emphasizing that if you want to compare this work to some other work that's done on equivariance and so on, that work is often being done from a syntactic angle. So let's pause to state the central claim of categorical deep learning. The proposal is basically that a neural network layer should be viewed as a homomorphism between 2 algebras for the same endofunctor. The endofunctor describes the kind of computation a network needs to respect, be it a group action, a list fold, or an automaton transition. And the algebras describe how that computation transforms the specific data. The homomorphism is then a function that maps between these 2 data representations while preserving the computational structure. When this homomorphism is a group action, you recover geometric deep learning, but the framework itself is far more general.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.