Sarah Saab: belief

Transcript context

…There is a real challenge to align AI models with what we want to do. Right? And there was actually a paper that you pointed me to from Anthropic called Agentic Misalignment. Massive kudos to Anthropic for releasing the entire methodology quite openly, I think, and transparently. They're tackling the same problems all of us are at the frontier these days as far as human values alignment. They gave AI systems a goal of working towards the benefit of The United States and this fictional company, and then gave, these agents access to email, email accounts for some of the c suite of this fictional company. Long story short, the AI agents found a notice saying they were gonna be decommissioned and also a notice, not a notice, but an email showing that the person intending to decommission them was having an affair. Totally independently of any prompting, all the major frontier models derived solution that involved blackmail, essentially. And the problem with these goals are they are quite abstract. Right? You know, they're open to interpretation. And you can change the goals and and you get different amounts of misalignment. You can even remove the goals and you get different amounts of misalignment. So this rather proves the point that how do we actually kind of communicate intentions to these models and how do we know that they're going to follow our instructions?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.