Tim Scarfe: belief

Transcript context

…places where they're better, places where they're worse, places where they have surprising gaps. So it's different, but but I wouldn't say that there's a fundamental lack of composition there at all. I think if if anything, the biggest gap between transformer based models and what we do is actually narrative memory, right, or being able to form long term memories and and and that way have a kind of persistence of a self over over long periods of time. They they don't have that yet. I'm conflicted. You're pointing to this universal representation hypothesis. I think Chris Ola popularized it with some of his visualization experiments. And it it's true. The representations are very convergent. And and other things lead me to to believe that the models produce these kind of superficial imposters that they give you exactly the right answer, but for the wrong reasons. And 1 of the hints of that is when you do variations on on the input, it's it's not robust. There's the there's the Turing machine argument as well. So that, you know, these LLMs are finite state automata, but they can access tools which are Turing complete. So, you know, perhaps we could say the system is Turing complete, but I don't believe that ChatGPT is is effectively searching the space of Turing machine algorithms. It hasn't been trained to do that, But it is surprisingly robust with the ARC challenge. It can actually Yeah. You know, it can it can do really well, especially if you do some evolution and do do some refinement and so on. So it feels like we're we're knocking on the door, but it but there's something missing. I think that in many of those cases, we're not doing the we're not doing a fair human comparison. You know, we we often, you know, and this is a little bit similar to our illusions about knowing how bicycles work and so on. You know, we I I hear a lot of people, you know, say things like, well, you know, but but look at this case where we just flip the logic, you know, you know, we we change it from do to don't and then, you know, it gets it wrong 30% more often and so on. You know, my my first question is always, have we done the human baseline? And and it turns out that surprisingly often, the human baseline shows the same the same property, you know. And this doesn't mean that humans are incapable of doing, you know, the fully robust, fully general version of these things. Right? If you're a logician or if you think about it carefully, you know, you can you can really write down your premises and be super robust to, you know, flipping the knots, you know, in the way something is is formulated. But most of us don't operate that way most of the time, you know, and we're highly susceptible to logical illusions, cognitive illusions, etc, which turn out to be in many cases surprisingly similar to the to the machine case. So I'm I'm I'm kind of unmoved by, you know, by by a lot of those and I I think often often we're we're being a little sloppy about how how we do it. It's certainly the case that that, you know, transformers don't aren't searching systematically over all possible Turing machines. I mean, we don't know how to how to do that. You know, you you have to take shortcuts of various kinds in order to make that that whole problem of of induction over over programs computationally tractable, whether you're a brain or a, you know, or a transformer.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.