Max Bennett: uncertainty

Transcript context

…Yeah. It it feels to me that there is a bright difference between, let's say, a transformer and and the neocortex. And I think the difference is, maybe agency is not the right word, but that you can think of the the neurons, I think, as having some kind of autonomy. So So they're sending messages to each other and then the other you know, it's eventually consistent. So the other neuron will get the message and it will decide itself what it's gonna do. And in a transformer, just because of the way they're connected and the back prop algorithm and so on, they they all they all kind of ride a Mexican wave together to use an analogy. So that it feels like a difference in kind to me. Well, mean, clearly clearly, as I, argue in the book, I don't think that the brain is just 1 big transformer. So I would agree with you. In in the human brain, unless you think there's something that's nondeterministic and sort of magical happening, you know, I think you would still say that there is either, you know, base firing rates of neurons, that and then there's sensory input that flows up and then goes through the brain until eventually there's sort of it's affecting muscles until you're responding. So there is, you know, there is sort of a deterministic flow happening. It might not be as feed forward as what's happening in transformer which is definitely the case, but they might both be sort of, you know, deterministic in in a similar fashion. I think a lot of people I've had this exact sort of argument with a lot of people and 1 counterargument that people have towards this idea, I don't know if I fully agree with it, but it's interesting, is that attention heads really are doing something more magical than we give them credit for, which is they are kind of dynamically rerouting and effectively resetting the network based on the context that the prompt is getting. And so although technically it's just a series of, you know, matrix multiplications, etcetera, if in principle what's happening is these attention heads are doing something really clever where they're looking at the context of a prompt and then effectively dynamically reweighting the network to decide what it cares about and what it doesn't. So there are people that think, you know, there is something really interesting happening in the transformer that might be analogous to certain things that are happening in the brain. But, yeah, clearly, you know, these these feed forward networks are are not capturing everything that's going on in the brain. Yeah. It's really interesting what you said because the way I read that is things like chat GBT and language models, they are entropy smuggling or agency smuggling. So what what that means is they kind of just do what you tell them to do and all of the kind of, the agency, so my directedness comes from me. So I give it a prompt, it does the thing that I wanted it to do And then the kind of the the mapping that you were talking about, I interpret that a bit like a database query. So, you know, depending on the prompt you give it, it'll activate a certain part of the representation space, and it will give you a certain result back. But the the the brain has this thing where all of the neurons have their own directedness. And and the weird thing is at the cosmic scale, agency is a site it seems to emerge. So even transformer models that were acting autonomously could presumably in large enough scale give rise to something that we think of as directedness or goals or purpose or or or whatever. But it's almost like, in the natural world, because there are so many levels, scales and scales of independent autonomous things just kind of mingling with each other independently, and then, like, downstream mixing their information together and rinsing and repeating over many, many different scales. That seems to be the thing that gives rise to all of these amazing things like agency and creativity and and etcetera.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.