Petar Velichkovich: evaluation

Transcript context

…There's a historical analogy worth keeping in mind. Before the periodic table, before we understood protons and electrons, practitioners of alchemy made real advances but without a principled foundation. Deep learning today may be in a similar position: we have powerful empirical results, but we lack the fundamental theory that would let us derive new architectures rather than just stumbling upon them. Categorical deep learning is an attempt to find that periodic table for neural networks. Deep learning, despite its remarkable success, is a field permeated by ad hoc design choices. Neural network architectures have all these knobs and tweaks that we can't formally justify just yet. There is no unifying framework for deep learning. There is no unifying framework that would that would explain the probabilistic perspective, the neuroscience perspective, and merely just the gradient based iterative updating perspective. In fact, in the future, we might look at deep learning very differently. And our claim is that category theory will become the unifying deep learning framework. But you seem to be making the argument that the interpolative function space of neural networks can model algorithms more closely to real world problems, potentially finding more efficient and pragmatic solutions than those classically proposed by computer scientists. We are working in this high dimensional space, which is not necessarily easily interpretable or composable because you have no easy way of saying, for example, in in theoretical computer science, if you want to compose 2 algorithms, you're working with them in a very abstract space, which means that, you know, you can easily reason about stitching the output of 1 to the input of another, whereas you cannot make that easy of a claim about latent spaces of 2 neural networks. Right? Geometric deep learning is powerful, but it assumes all transformations are invertible. What happens when computation destroys information?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.