Cameron Berg: belief

Transcript context

…iously needs to be modeling and processing that, but maybe in addition, it needs to be modeling something about itself in relation to that context object in order to interact with it in the right way. and i think you know felix binder and a couple other folks did really interesting work along these lines basically demonstrating that there there there's probably something like a self modeling or maybe self awareness they would think is too far. But there's really some flavor of this going on inside inside LLMS, which I think was some of the most interesting early work on introspection in LLMS. What is the name of the paper? Tell me about yourself. They, they did a couple things here. And I think OE and Evans was working on this too. One of the papers was showing that another model basically trained on the same data that one model is outputting, cannot predict that model as well as the model can predict itself. Basically holding all the relevant things constant that you'd want to hold constant to make a claim like that. And so that's it's like there's some sort of privileged information that models have about themselves. and then yeah this this other paper i'm not remembering the exact details but but my basic conclusion if you sort of take it on some level of faith from felix 's other work here is there's probably something like a coherent self modeling engine in these systems that seems to be maybe like interest instrumentally selected for when you're doing something like really good next word prediction across long horizons in a way that's supposed to be helpful to a user. hat seems to be maybe like interest instrumentally selected for when you're doing something like really good next word prediction across long horizons in a way that's supposed to be helpful to a user. This to me, I think it's basically like what you're saying. I don't think our just so stories are very different. But yeah, I mean, again, we can take a step back and just like a lot of interesting cognitive properties seem to emerge slash come along for the ride when you train systems on every cognitive linguistic output humans have ever bothered to write down. Maybe that's not that crazy and spooky. Yep. They're pretty good at theory of mind. They're really good at having working memory style dynamics. They're really good at selective attention. And yeah, maybe they're really good at something introspection. Like people bristle a little more at these because it, the whole consciousness question comes into view. But I don't think it's like at the most general level, intelligence came along for the ride. and we still you know philosophers still maybe don't have a crisp super rigorous sort of intelligence is this thing here's how to test it here's how to model it here's how to understand if a system 's simulating it versus actually having it. And we just sort of blew past it pragmatically, empirically. We have systems that are brilliant by any reasonable metric. And, and, you know, I have no patience at this point for folks who are still on the sort of stochastic parrot wave. This to me is just advertising being out. have you talked to claude opus four point six? It's as intelligent as any reasonable definition of intelligence. These systems are intelligent. And I don't think it's that wild to think that something like consciousness could come along for the ride in a very similar way. We don't have philosophical certainty about it. People point to slightly different things when they talk about it. You build out a cognitive system that's sufficient, sufficiently sophisticated and capable. It may be that cognitive traits that we see in every other cognitive system, meaning, you know, animals we believe are complex animals, everyone is pretty confident, are conscious. We're pretty certain humans are conscious. We build sufficiently advanced systems, they might, those properties might just come along for the ride without us. The universe, I think Neil deGrasse Tyson says, like the universe does not need your permission to continue unfolding. Like consciousness could just be a complex property of cognition. Us not having a good model of it doesn't mean reality is going to wait up for us to build that model for it to start getting, you know, accidentally instantiated in these systems. And that's the absolute most basic story I think I can. I can tell along these lines, reality. Doesn't have to wait for us to have a good model.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.