Ilia Shumailov: preference

Transcript context

…it takes time. Yeah. I mean, I suppose it's related to this idea that we are now deploying agents into production. Are you? I don't know. A a lot of people are. And the these agents have an incredible amount of intelligence and flexibility, which means we don't precisely specify what they do. We just give them kind of abstract guidance, and they have the flexibility to kind of adapt to novelty and and just do whatever needs to be done in the situation. But but then, you know, you have this whole reward seeking thing, which is that they can just suddenly, you know, adopt a model of the reward environment, or there might be conflicting instructions. And but, basically, for a myriad of different reasons, they might do the wrong thing. And we need to have sophisticated systems of monitoring just to understand what they are doing. I don't know. Sounds very complex. I don't know. I think the honest answer is yeah. I think we need an experiment. Science, science, science, scientific method. Very well defined questions, very precise outputs. Like in our paper, we openly say, although this is anecdotal evidence at best, we have collected some data. We can make some sort of observations from it, but that's the extent to which we can reason about this. I don't know, unfortunately. We'll see. We'll see. I think the more scientific experiments we do, the more sort of meaningful assessments we can make. But is it 1 of those things where, yeah, we don't have a good theory around this yet? So, you know, maybe we're not talking about it using the right concepts, but clearly, bad things are happening.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.