Tim Scarfe: preference

Transcript context

…I I don't disagree with that, but I don't why why wouldn't we be able to find that algorithm for neural networks? Right? Like, why wouldn't, you know, we we train, neural networks much bigger than the brain. We put a lot of compute towards them. Do you you just don't think that, finding the same algorithm is possible with with SGD? I think I think with SGD because the fascinating thing is that, you know, if you look at all of the FSA algorithms, a a tiny sliver of those algorithms are capable of controlling, you know, a Turing machine and expanding their memory and so on. So it's in the space. And I don't know if you saw that amazing paper by Kenneth Stanley, the fractured and tangled representations paper. And he had this beautiful diagram and he said that, you know, SGD finds the algorithms over here and neuroevolution algorithms find the ones over here. And it just so happens that the neuroevolution algorithms find ones that have these factored, you know, representations, which means they they find representations that are about the world, that are grounded in the world, that carve the world up by the joints. And if only we could find those things. You know, when I spoke to Schmidt Hoover, he said the same thing. He said, like, you know, it is actually possible to find the right weights in a neural network to make it, you know, effectively Turing complete with some caveats and so on. But when we do SGD, because there are all of these shortcuts, right, it is a bit like good hearting. It it will always just find the wrong thing. I need to think about that a bit more. Okay.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.