Jeff Beck: belief

Transcript context

…We should also introduce this term transduction. So my definition of transduction is that you're actually doing search or optimization as a function of the test samples. Like I interviewed Clement Bonnet, had a VAE on ARC, searching latent spaces. And he actually searched through the decoder as a function of the test sample. And because these models, they are maximum likelihood estimators, right? Which means they're always giving you a kind of smoothed out average. And there's so much information in the test sample. Let's just riff on the relationship between energy based models and Bayesian inference. So of course, they have this advantage that you don't need to do this for expensive intractable normalization. Yes. Yes, tell me about that. My take on it is is that an energy based model and a Bayesian model have a lot in common. Right? In many ways like energy, I mean, well literally in physics, right? Energy is like log energy is log probability. Now, of course, there's the normalization, you know, factor that you don't need to worry about if you're just doing if you're just minimizing energy. And so the difference between, you know, like which is sort of like, you know, in a Bayesian framework, that's like saying, well, know, I'm not actually gonna treat some of these latent variables in a probabilistic way. I'm just gonna do maximum or map estimation on some of my variables and just be okay with that. And that's 1 way to interpret the relationship between an energy based model and a properly Bayesian model. There's there's a happy medium here, though. Right? And the happy medium is you can still treat it as if it's, you know, you know, you don't have to just minimize the energy function. But you can calculate the curvature down there too, do a Laplace approximation, and call yourself a Bayesian again. Right? Yes. There is more computation involved, but we've got a lot of great tricks for making that totally tractable. What's the relationship between the free energy and the free energy principle and the energy and energy based models?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.