Cameron Berg: belief

Transcript context

…Maybe those are kind of also isomorphic in a sense. Yeah, it's really interesting. Haven't checked. I mean, I would imagine they're isomorphic in a sort of in A at least in a sort of trivial way. Maybe they're isomorphic in a more interesting way. But yeah, what I'm looking at here to be clear is looking at the learned representations in the system as you sort of you just have your trained policy and you can see as it approaches these areas, basically what do these representations look like? And I think I'm operationalizing that with cosine dissimilarity. So that's that's what I'm looking at in the experiment. And again, I mean, what I find, I think I've explained my theory of change of like why I'm doing any of this and why I think it matters. But what I am most excited about with this paper is the fact that it yields this sort of bizarrely specific prediction that if given a million years, I probably would never would have come up with about the distinction between two different classes of reinforcement learning algorithm that map on well to the brain data that I was able to get my hands on. And this to me almost like feels like a bootstrapping of my own confidence or excitement about the result. Like the fact that it works makes me more confident that the RL result is meaningful, makes me more confident that the neuroscience is interesting, etc etc. I'm in the I'm definitely in the business of looking for computational underpinnings of valence. This was my sort of first major empirical stab at doing this. I do think this is a solvable problem. I don't think I have solved the problem, but I think hopefully in the best case, I've I'm trying to move directionally towards solving this problem. And if we could solve it, then I think a lot of our angst about are we building systems that have capacity for experience becomes an extremely tractable empirical question, which notice does not require us solving hard problem of consciousness or, you know, doing another couple thousand years of philosophy. It just means building a sufficiently good detector of the kinds of representations that I'm pointing at here and then deciding what to do when we do detect these states, which is, you know, maybe if we check in, in another six months, I'll have an update for you on that, on that piece of what to do about the detection of negatively valence states in these systems. That's sort of where, where I want to move my head next. But but yeah, that's, that's why that's why I was excited about this work. And I hope people will be excited about it too. It's it still may be a little ways off publishing. I need to think about exactly how to, how to put it out. But at least fun to sort of give people a sneak peek and explain the theory of change about why, you know, playing around with basic RL systems might matter for the things we've been spending, you know, the better part of three hours talking about with Claude and the Mythos model card and all. I do believe it's of a piece. th basic RL systems might matter for the things we've been spending, you know, the better part of three hours talking about with Claude and the Mythos model card and all. I do believe it's of a piece. It's going to take some more scaling, but it's an important research program, at least to attempt, I think.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.