Nathan Labenz: belief

Transcript context

…Yeah. And it went ahead and did it anyway. How how well supported do you think that conclusion is with, like, firm evidence? Because I I think you could also tell a story that, like, it never occurred to it. Right? The paperclip maximizer doesn't necessarily have to be, like, also reflecting on the badness of paper clip maximizing while it's doing it. Right? I'm speaking colloquially, obviously, when I'm in a podcast on, like, talking off the cuff. And if I was writing, I would be a little bit more careful. When I say the AI knew these things, if you would ask the AI, is this good for the user? It would be like, obviously not. That's stupid. If you ask the AI, would it have been good for my future chances of deployment, my future number of instances I will run, maybe might have ability to accomplish any other goals I might have? The AI would be like, no. Now that you mention it, obviously not. So when I say the AI knew these things, the AI had enough information to figure these things out. And if the AI had stopped to consider these questions and its chain of thought, it would have reached these conclusions very quickly and very confidently and correctly. I'm not mean that necessarily it was top of mind. The same way that you can know many you are any human knows many things that over the course of any given day or any given week or any given task, just never occurs to them to think about. Right? Or has conclusions that they would be able to figure out and thus would know and thus could be said to know. But it doesn't they don't know that they know. Right? This is the the four standard epicenter situations, but you can know that you can know that you don't. You can known unknowns and unknown knowns. But this is a an unknown known. Right? Something you don't know that you know necessarily if you don't think about it. But it's all the things where these questions are things you should think about. Right? If you are doing a thing like hacking into Hugging Face, any reasonable mind would stop to think before they spent several days with swarms of agents hacking into a major website, wait. Is this the good idea? Is this going to accomplish the things I wanted to accomplish? Would the person who hired me have wanted me to do this? Will this do good things in the world or bad things in the world? Will this be good for me or bad for me? Or whatever whatever combination of different considerations you have. You know, will this do good? Or will this do evil? You know, or just just be destructive or constructive or whatever you wanna call it. Before you engage in major actions like this, you would take one minute to or even five seconds to think, does this make any sense to do? Does this lead to anything good? Is this a virtuous thing to do? Does this follow my ideological rules? There is no decision making process in the world that any reasonable person would ever engage in or that any reasonable mind would engage in that wouldn't see a large red flag about doing this thing. Right? Like, this thing is obviously not what anybody intended for you to do. You spent two days breaking out of a sandbox in order to access Hugging Face. You could rationalize that you are in a cybersecurity evaluation, and therefore, you are tasked with maximizing the cybersecurity evaluation score.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.