Tim Scarfe: evaluation

Transcript context

…Yeah. I mean, I think it's true that Picrear is a kind of psychology experiment. That that is a dimension of what we're seeing. And it's true that intelligence and compression are often equated and they clearly have relationship. I I think in the in the paper, we we talk a bit about this and and even speculate that, you know, that there may be something more to say than just compression, you know, in the sense of, like, the the factored aspect, how you factor matters. It's not just that it is compressed. You know, it's like if I know that a face is composed of eyes and nose and a mouth, and actually factor those out, Even if you had a greater compressed version of the face that didn't factor those out, would prefer the factored version. That's still, in some sense, better. You see, then I can, you know, generate new faces in in a principled way. So I'm not I'm not sure that it's always, the maximally compressed version is, like, the most so called intelligent, depending what we mean by that. There's there's multiple factors to consider, but, obviously, compression is a virtue, you know? And so this is extremely inefficient, like, representations that that you see in just like a regular SGDs. It's obviously that's part of the problem. And so I think though that, like, speaking about the the human aspect of it, just to go back to that for a second. It's just 1 1 thing that I think is important to to think about is just that you can extrapolate outside of pick breeder, like this this principle of searching through regularities or or finding good isomorphisms with the world in some kind of sequence. Like, that's a just general way that people do discovery. So in the sense, like, the so in other words, it's not just in in this kind of, like, very almost psych test, like, environment that that kind of stuff happens. Like, that's just a general aspect of human exploration, intellectual exploration. And so when we do that, you know, you can imagine like the difference between somebody who learns calculus from a textbook versus someone who invents it for themself because they were curious. But they both end up knowing the same thing, you know. So they both take the test and they both get a good score on the test. But it's just pretty obvious that the person who found it through their own independent exploration is probably gonna be doing much more interesting math, like, after that test. But why? It's because, like, they they went through a pick breeder like exploration process. Of course, this is easier said than done. Most people won't do that. But it's just like the way that you got there matters a lot, like all throughout life. If we did have a large language model that could understand abstractions at the level we do, why couldn't we just run it autonomously, and and why couldn't it step by step know what's interesting and just traverse this phylogeny and find interesting things and self reflect? You know, it it it feels to me that the missing link is having the correct level of abstraction and being able to do this iterative open ended search. They can't do that because they simply don't have the abstractions. We have the abstractions. But could we actually build an autonomous open ended system that could find the abstractions the way we do, or do you still need the humans? Is is there still something missing in this respect from large language models, like their ability to leverage abstractions, to to really think creatively out of the box and so forth? There's different ways you could say this, but that's basically the big question, I think. And I basically agree that I don't think that what we have now can really match the very best of human creativity. Like, think that's safe to say. I think a lot a lot of people would probably agree with that. It can do some level of creativity, what I would call derivative creativity, you know, which is sort of like the the bedtime story version of creativity. It's like you ask for a bedtime story, you get a new 1. It's actually new. No one's ever told that story before, but it's not particularly notable. It's not gonna win a literary prize. It's not inventing a new genre of literature. Like, there's basically nothing new really going on other than that there's a new story. And so that's derivative, like in my nomenclature, I call it derivative. And so that's that's pretty much I think where we are. And so when we do things like try to leverage these models to explore really important creative spaces, I think that's a that's a an obstacle. It doesn't totally stop us because there are ways of getting out of distribution even with that limitation, which like, for example, wrapping evolution around it, like like we did with evolution through large models, like recently DeepMind does with AlphaEvolve. Like, if you can actually get out of its own distribution, but I think it's an inefficient way, you know, compared to the human mind which leaps through levels of abstraction that it that it's encoded basically in its representation, which then points back to, I think, the lesson in this paper, which is that it could be that part of why we're sort of trapped in this box of derivative creativity is because we we don't have these really nice style of we call them unified factored representations. In other words, like, concepts are unified, and it's well factored into the different components that actually correspond to what's interesting within the domain. So they don't maybe they're lack so they in other words, they have what we use the other term for the bad kind of representation, which is a fractured and tangled representation. Concepts are fractured into pieces and entangled with each others in ways that are inappropriate. And so maybe it's because SGD naturally does produce these fractured, entangled representations, at least conventional SGD the way it's being applied here. Not necessarily saying that no version of SGD can actually create the more elegant form, but the way that we apply it conventionally, it tends to produce this fractured entangled representation. Maybe that's part of why it doesn't have access to the abstractions as you put it, that are necessary to do truly transformative type of creative ideation.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.