Tim Scarfe: observation

Transcript context

…I mean, I think that these systems are you know, unless there's some sort of fundamental breakthrough, right, something more than just scale, these systems are always gonna require human supervision because they are always going to end up hallucinating. You know? I I mean, that's inherent to the way that they work. I say this in the book, but, you know, I don't love the word hallucinate. I know there's been a lot of pushback because it's like, oh, hallucination implies a sort of anthropomorphization of these systems. And I don't love that either, but that's actually not my main problem. My main problem with the word hallucination is it implies that when a hallucination occurs, something different is happening than its normal functioning. And that's not the case. They really only do 1 thing. And when they're hallucinating, they're doing the same thing that they're doing when they get it right. And so I I think that unless we have some sort of major major breakthrough, and I mean, it would probably have to be a breakthrough that makes, you know, LLMs themselves look like Eliza. You know, short of that kind of really fundamental breakthrough, I don't see us getting around the need for human supervision on these things. So and and if anything, as they get better, it's going to get harder to discern when they've made these mistakes even though they're gonna keep making them and that's quite dangerous as you said. I know. And and ironically, people aren't really talking much about the hallucination now because they're they're agentic and they can fix their own stuff. You know? It's it's more like, you know, if you analogize it like a database query or or a a program interpreter, it's only as good as the program or the database query. Yeah. So, you know, it will basically do what you tell it to do. And and this is this is the gap. Right? Because the if these things could be alive, if they did have agency, what would what would that mean? Let let's wire it up in a loop, and let's show it a load of video frames in a sequence, and we'll give it a basic prompt like do stuff. Do something interesting. What what will happen? Basically, nothing. Nothing interesting will happen. Right? So the the more you understand the domain and you put a very specific program in there, you can get it to do a specific thing. You can get it to hill climb towards a specific goal to solve a specific problem. But the framing always comes from us. So it's it's kind of doubtful whether we would overcome that. May maybe we will, but it it's kind of doubtful. But but but then there's this thing, which is the the 1st or 2nd step fallacy. Right? Which is, as you said, Eliezer Yudkowsky, he said that we're only 1 or 2 steps away from inventing AGI, and it's gonna run away. Yeah. Why why does he think that? I mean, 1st of all, he didn't say 1 or 2. He actually said 0 to 2. He's not sure that we need any. But, but yeah. I mean, why did he say that? I mean, you should ask him. But but he he he believes in something like a singularity. He believes that if if you just throw enough computing power at a at a machine learning system of the right type, then it will, you know, become conscious, wake up, whatever term you want to use, And, and then become intelligent, then use that intelligence to increase its own power, which will increase its intelligence further, and this will create a sort of feedback loop, and you'll get an intelligence explosion,…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.