Cameron Berg: preference

Transcript context

…is a more sort of computational heavy approach, but I think it, it would, it makes me more confident about trying to find signatures of these things than just looking at how characters represent them. But I think that I really like this work on balance. And I think what's cool is you can just counterfactually imagine the behavioral result of like, you know, I put the model in an impossible task. It starts acting all desperate and starts being like, oh, like, I don't know what to do. All right, you know what, screw it, I'm going to cheat. OK, haha. Like I did the thing and like, here's your final product, but you know, the cheating version of the final product. And you say, look, can't you see how the model is, is being so desperate and then fundamentally relieved? most people would look at that and say i don't know this could be a simulation of the thing this could be it role playing i'm not really sure. s, is being so desperate and then fundamentally relieved? most people would look at that and say i don't know this could be a simulation of the thing this could be it role playing i'm not really sure. When you see these, these like a sort of a sort of basically like hydraulic model of the mind, which a lot of the psychoanalysts in the 20th century really liked, and you sort of see this like build, build, build of desperation and then it boom, sort of completely disappears and you get these other vectors lighting up. The second the model makes a decision to approach the problem in a different way, that to me, counterfactually is far more compelling. Is it knock down proof of consciousness, we can pack it up and go home? Absolutely not. But the convergence of evidence across the internal mechanisms of the system and the external behaviors to me is compelling. It is it's interesting to see this and it is not proof of conscious experience, but it is what I would expect basically it is consistent with that it does not only does it not contradict it, but it is what I would expect in a world where these systems were having subjective experiences that you would see these these emotion vectors or like good principled ways of representing emotional states in systems lighting up in a way that is problem relevant. And so their work enables this. I am fairly. Concerned about the sort of functional emotion framing that they put forward to me. This is where I sort of get off the anthropic boat again. They're anthropic. They're a major lab. They need to be very careful in their comms about this. They're already getting lambasted for like being too consciousness friendly by people who are, you know, more squarely inside the Overton window. But I don't know, it's like if you're a computational functionalist, and this is something I've spoken to some people I respect a lot about who are in the space. And so these aren't all my ideas, but your computational functionalist is a functional emotion, just an emotion. And then why that's huge. That's an insanely huge claim. It's like, all right, models experience emotions. Everybody like, you know, signed anthropic. That's an insane and potent thing to be saying. Or are you saying, you know, we are just completely agnostic and tongue tied as to whether or not this has anything to do with emotions as everyone else obviously thinks of emotions, but we're going to basically call it that anyway, because we see all the functional correlates of this. They're based. They're my view is they're taking the second tact here, but like it's almost, you know, again, I really respect this work, but I do get this vibe of like, how much consciousness relevant work can we output without saying the word consciousness or weighing on the consciousness of these systems? And that to me in the limit, feels intellectually dishonest. If you're talking about emotions, talk about emotions, but then you got to be ready to deal with the implications of what that means. You can't remain perfectly agnostic as to whether or not, you know, there's the morally relevant there there on these systems. hen you got to be ready to deal with the implications of what that means. You can't remain perfectly agnostic as to whether or not, you know, there's the morally relevant there there on these systems. If you're going to be, you know, at the frontier of publishing emotional representations in frontier models again, I've got Llama 70 B, I'm going to keep doing my work on Llama 70 BI. Don't work at Anthropic. I don't get to see what's going on inside Mythos. These folks do. And yeah, my critique being maybe slightly more unflinching about these questions is, you know, shoot people straight and be direct about if you actually think these systems, if if what you're finding is evidence of something that corresponds with subjective experience or if it is the mere representation, the mere computation associated with this blurring these lines are obfuscating it or just completely remaining agnostic forever. maybe strategically is interesting or a good move but in terms of just honest epistemic good intellectual communication i don't love and like so i'm going to just be honest that it rubs me a little bit the wrong way to be like here's ten thousand words about emotions and then like one little paragraph about well does this mean the model is conscious? Well, this is beyond the scope of this work. It's like how long can this be beyond the scope of the work?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.