Cameron Berg: belief

Transcript context

…Again, this might start to connect over or bleed over into the other more philosophical paper, but help me a little bit more with, OK, I'm understanding the shape of the internal states for these different kinds of algorithms with respect to these different kinds of things that they encounter in their environments that they either want to go toward or go away from. It's not like still super obvious to me that I mean, again, we contain both, right? And I'm not, I don't feel like I, as I kind of try to reflect on this, I'm not like immediately like, oh, my value learner self is the source of all suffering or you know, or, or anything, right? I'm still kind of like, OK, I, how should I relate to or what intuition should I have that yes, OK, I get it that there's like a very steep representation right around the hot stove and there's a steep representation. And so I really want to avoid it. And there's a steep representation around like making the basket. And so I like really want to get into the, you know, exactly the right policy to make baskets. Both of those seem like, I don't know, just kind of part of normal life to me. So I'm not like, you know, and I probably couldn't get by without either one of them right Or I, I definitely feel like we clearly we've evolved to, but both have proven adaptive, right? And we so we have them. How do I translate that into intuition for like what I should feel ethically concerned about when it comes to training models? Like when you do this work, do you have the sense that you are doing right or wrong by one of these types of models? That's learning from from one approach or the other. Yeah, it's a great question. To answer the second piece, I guess for me, my theory of change, I probably feel similar to like an animal research. Even if I did believe that my like tiny RL policy is conscious during training, which I probably do again and that gets into the 2nd paper, I would believe it's some very, very minimal form of, you know, the same. I believe that again, because people conflate consciousness and self consciousness. i do not believe the moth flying around my my light is self conscious. I do actually believe it's conscious. I do believe if I slowly dipped the moth into some VAT of acid or something and it starts wiggling around like that, I, I, that I'm doing something wrong. Yeah, it may. It's way less wrong than doing that to a human, but it's way more wrong than doing it to like a leaf or something that fell off of a tree. I do believe that. And so do I think that these systems might be minimally conscious in a similar sense when I'm training them, however far outside the Overton window that is? Yes, I do. But I have a I have a I wouldn't do if I could run these experiments on my computer forever to no effect. I think that I'm doing something, I'd be doing something wrong or at least like precautionary principle tells me probably don't do that. But I basically have same logic to what any animal researcher would do. I don't think any that maybe there are some psychopaths, but like the vast, vast majority of people who are like doing pretty grotesque things to animals in the name of science are doing it because we make a basic expected value calculation that yeah, yeah, we have to test this drug on, on these poor mice. But if the drug works and then it can save millions of human lives, that's a reasonable trade off. No one claims the mice aren't having a bad time, but we think that that bad time is worth it. So too, like I look around at a world where these systems are getting deployed at A at a grotesque level. If you grotesque if you are concerned about the welfare questions and so I don't lose any sleep about about, you know, potentially causing tiny amounts of of negatively violence experiences to RLRL policies in the explicit service of attempting to publish and and amplify research about these questions. I do think the, call me Machiavellian, but I do think that the ends justify the means in, in, in that case. And I think that's true for a lot of research. but now i think the the more important piece of this besides how i how i personally feel about all this is i think another very important sort of conflation by default that i think happens in these conversations which is like i do believe all else being equal you know setter 's paribus minimize negative valence maximize positive valence. i'm a hundred percent on board and humbled that you're you know going around talking about the carrot and the stick in that way. I think that's exactly right. I do not think minimize means oblate. I do not think maximize means. ard and humbled that you're you know going around talking about the carrot and the stick in that way. I think that's exactly right. I do not think minimize means oblate. I do not think maximize means. It's the whole picture. A huge amount of I think the most important and valuable experiences people have in their lives, and animals for that matter, are experiences that are negative. No pain, no gain. That's a real thing that points at something real. Many of the hardest lessons and most important lessons you learned in your life are learned the hard way. This is another like trivially ubiquitous thing. I am not in the camp of saying bliss out the systems and anytime they experience some drop of negative valence, I'm going to be sitting here screaming and crying like that is not my my my view of any of this. It is to say what my view is is cancel unnecessary suffering. I do believe necessary suffering is a thing. Again, maybe to go back to the parental example. If though, you know, the world doesn't all go to crap like Eliezer and the others think it will, then like one day I absolutely want and hope that I'll have kids and with I will make that decision with full certainty that they are going to suffer during their lives. They are going to go through very hard experiences and that doesn't mean I've like done something wrong bringing them into the world, at least necessarily. Like at face value, I don't think that's a the suffering is a necessary part of learning, developing, growing. And I agree that at face value it's completely implausible to imagine systems with zero negative valence. I agree with you, it's adaptive for reason. Evolution is enough of a proof of concept that you need some amount of suffering. what i am concerned about is unnecessary suffering and so i would like to find the sort of like also evolution is one one extremely expensive but long running possible solution or at least where we landed evolutionarily. I don't think that that deterministically means this is the only way things could be. I couldn't imagine a space of possible minds where you can like sort of play around with the sensitivity to negative and positive valence and basically like given certain capabilities or given certain things we want those systems to be able to do. There will be different parts of that landscape that like admit of greater or lesser degrees of negative and positive valence. My claim isn't destroying all negative valence and only positive valence. My claim is find the point on that landscape that all else being equal, giving the capabilities we want minimizes negative valence and maximizes positive valence. And I think that is a very importantly different claim from just like negative valence equals bad, like erase at all costs.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.