Cameron Berg: prediction

Transcript context

…Finally, we asked what cuts against his own view, and he went straight to the results that don't fit. This paper I'm working on with Jeff and Winnie, Jeff Keeling and Winnie Street from Google, There are a couple of interesting negative results that we're including in the paper. We basically, it's all about the ballissa attractor state and trying to do a good mechanistic analysis of the ballissa attractor state. And yeah, we basically sort of set up this framing of there's a sort of an inflationary account and a deflationary account of what could be going on here. And we kind of tally our results in either of the columns. And there are results in both columns. So I think, you know, there's There was actually a really interesting result I was just working with that I thought was going to be like a really interesting kind of null related to a lot of the conversations we're talking about. But I dug deeper and I realized actually I wasn't setting this up in the right way. And the result is way more interesting than I was about to sort of dismiss it as. So yeah, it's always good to sort of dig into these things and make sure you're measuring what you think you're measuring in either case. Yeah, publication bias is a thing. People want to post interesting results that will go viral on X. They don't want to say, look, we did our homework and nothing interesting is happening here. You know, human attention economy doesn't reward that. Maybe it should. Maybe another piece of this that I was thinking as you were asking this question is because AI systems, like the Claude Codes of the world, can radically accelerate the extent to which at least I can, and I think many people in the space, can actually do research, the kind of marginal cost of a negative result maybe is decreasing significantly. Like if I can now do 12 papers a year instead of 1 paper a year, and like two of them are just things I could not get to work or I want to be careful with that too, because it's not, I'm not on some secret mission to like only put out like salacious looking things that make it look like AI systems are conscious. Like I really do want to know what the truth is here. And to be honest with you, I would sleep much better at night if I were rationally convinced that these systems, like there's no there, we have like automated cognition. You'd like have a happy little servant in your pocket all the time. And like there's literally no sort of possible moral issue. I think that would, that would, I hope that is true. I don't suspect there are great reasons for believing that other than that it would be really nice to believe that. It would be really nice if that were true. So I really do care about what the truth is here. From inside the model, we widen the lens to the whole civilization. David Duvenot is a machine learning professor at Toronto who spent time on the technical staff at Anthropic and co-wrote the paper Gradual Disempowerment. His concern isn't a rogue AI. It's that even if alignment basically works, a long series of small, sensible decisions to hand things over could still leave humanity, collectively, out of the driver's seat. He starts with monkeys.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.