Max Bennett: belief

Transcript context

…Interesting. Okay. I wanna close the loop on what what you said about the brain being an imagination filling in machine. So you said that it does filling in. It's 1 at a time. It can't unsee visual illusions and evidence is seen in the wiring of the neocortex itself you say so it's shown to have many properties consistent with a generative model. The evidence is seen in the surprising symmetry inseparability between perception and imagination that is found in generative models in the neocortex. And you give examples like, illusions and how humans succumb to hallucinations, why we dream and sleep, and even the inner workings of imagination itself. So it really seems plausible when when you kind of think of it in that way. Yeah. I think I think there's is a reason why, so much of the neuroscience community, I mean, what I'm saying there is not really novel so much the neuroscience community has sort of rallied around this idea of predictive coding, which is very related to active inference and generative models. Because there's just so much evidence that what's going on in the neocortex, the imagination of things, episodic memories, I mean there's been some good evidence that episodic memory, other words, thinking about past and imagine the future are in fact the same underlying process happening in the neocortex, which is again consistent with this idea that there's a generative model. If we look at the connectivity patterns, I didn't talk about this too deeply in the book because it's a little technical, but what you would expect from a generative model is backwards connections would be much richer than forward connections because you're modulating downstream. Course, NeoChorus is not perfectly hierarchical, but things that are in general lower in the hierarchy would have lots of inputs from parts of the neocortex that are higher in the hierarchy. It's absolutely what we see. So yeah, there's a lot of evidence that these are 2 sides of the same coin, which is there's some form of generative model being implemented. I do think in AI, 1 way in which this manifests is the very clear success of self supervision. I mean, idea although the actual predictive coding algorithms that people have sort of devised as neocortex implementing, when we've actually modeled them, they haven't outperformed any of the stuff going on in the AI world. The principle of self supervision, is can a system end up having really interesting emergent properties and generalize well when you only train it on predicting sensory input that it receives. And that clearly has become the case. I mean the transformer I think is a great example of if you just give it a bunch of data and you train it through self supervision, I. E. Masking so you hide certain data inputs, it becomes remarkably accurate and good at generalizing across data that it hasn't seen before, which is in principle all what people are predicting or claiming that the neocortex is doing as a generative model. Yeah. It it feels to me that there is a bright difference between, let's say, a transformer and and the neocortex. And I think the difference is, maybe agency is not the right word, but that you can think of the the neurons, I think, as having some kind of autonomy. So So they're sending messages to each other and then the other you know, it's eventually consistent. So the other neuron will get the message and it will decide itself what it's gonna do. And in a transformer, just because of the way they're connected and the back prop algorithm and so on, they they all they all kind of ride a Mexican wave together to use an analogy. So that it feels like a difference in kind to me.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.