Kenneth Stanley: evaluation

Transcript context

…This leaves us with a choice, the path of a singular goal orientated kind of optimization, which creates brittle fractured imposters or the path of open ended exploration, which ostensibly creates robust unified models. They argue that this choice fundamentally impacts 3 important things that we want from AI, which is to say generalization, creativity, and continual learning. If you think of the skull again as a metaphor for all of human knowledge, because that's what an LLM is trying to capture. It's not just a single image. It's like an image of all of human knowledge. For any input, it should output something that's convincingly human, then, like, it it could be just similarly an impostor. It could be the same. Underneath the hood, everything could be organized wrong, not the way you expect. It's like a giant charade. And again, this is very confusing and counterintuitive, like, for for for people because, like, you know, people are naturally, including even me, I would react like, but should I really care? Like like, when you say it's an imposture, but it's getting everything right in its human level. Like, what are you objecting to? But they but the point is that it can still be an impostor because it's like, what we care about here is not just that it's going to get answers right, like, good test scores, like, seem to be plausibly human when you talk about things that are in distribution. We want it to be able to go outside, like to do things that are creative, to be able to continue to learn, like to do to to get to the next level, including learn on its own and get to the next level. I mean, these are like the next frontiers for the field. If it's an impostor underneath the hood, then these kinds of things are are are going to hit a wall or become insanely expensive. It could be that you can always push through that wall, but the expenses just go up and up like crazy. Exponential worse, I don't know what it means, but it could be something terrible. We might already be seeing that. Like, you know, the amount of money that we're spending here raises questions. Like, is it necessary? Like, does it have to cost this much in energy and in money? So this flips everything on its head. Right? The very thing we're trying to control, the objective, is a bottleneck for the thing we actually seek, creativity. True creativity might even be intelligence. I mean, it's certainly the nearest quantity I can think of to describe what intelligence is.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.