Speaker unverified: belief

Transcript context

…I think it's possible because a neural network is is not too incomplete. So I I think in principle, can't do many of these things, but you can build a controller. Right? So you you could just build a very simple kind of envelope just as you did with your solution. So you you you had a bunch of code and it was doing this in, you know, this basically compositionality in code on top of the neural network substrate. And that gives you many of those things. You know, for example, we we often talk about library learning and library transfer. And I'm not sure if you've seen Eric Pang's solution. I'm speaking to him in Hong Kong actually in a couple of weeks. But he rather than the DreamCoder approach where they do this explicit library learning, He was doing it in a kind of implicit way using the LLMs. And I think there's a whole spectrum between you know, you don't have to do it explicitly. I think you can kind of use neural networks and you can do some kind of implicit composition and you can get many of these features. Also, I I want to say, generally, I speak about language models, I assume that they have basically a Python terminal that they can run. Oh, okay. So not so I guess 2 things. The first is I think if you have a large enough neural network, I think generally almost everything is you could you could represent a symbolic system, but of course, it's not Turing complete. But given a neural network plus the ability to write programs, then I think we're we're basically at the sis the we're at the human brain equivalent. So, yes, So that that is a hybrid system and that certainly is significantly more powerful. I'm just regurgitating my cohost to doctor Dagar because this is his, like, favorite point he always likes to make. But but he says that that's true, but stochastic gradient descent does not find the algorithms that allow the systems to behave as if they are Turing machines. God knows how it happened in our brains. There was some dint of evolution or something where, you know, we suddenly got the merge operator or god knows what happened and and we've got this this incredible, like, Turing complete algorithm in in our finite brain. And so so we're getting into that trainability thing. Yeah. You know? So, yes, maybe there is an out you know, there there is a set of weights that we might find 1 day and it can access like a Python, you know, tool and it can do all of those things. And what what you know, its capability now. Is it now effectively searching the space of Turing machine programs? I think it's not. Like, there's lots of problems there. Like, how how would it know which ones halt and which ones don't? And how would it be able to efficiently search that space? It feels like there's a gap now, but I agree with you that it's significantly stronger than not being able to use the tools.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.