Ilia Shumailov: evaluation

Transcript context

…And and why would it be possible to steal the models with this chain of thought? Can it be possible to steal the models with chain of thought? That's a wonderful question. Why is it possible? Yeah. Why is it possible? I mean, we know model stealing is a thing. Right? Like, model stealing broadly allows us by just simply quitting the models to learn the insights of the models, like, to learn the decision boundaries. And the best way, I think, to think about this is, like, in a more crypto cryptanalytic way. I guess, I don't know. I I guess the agents never done cryptanalysis. So, like, think about this as, like, you make minor minor steps in your input until you notice that the model behaves in a slightly different way. And then by finding exactly when and how it did it, you can kinda learn the decision boundaries themselves. And broadly, if you know the actual structure of the function itself, you can kind of fit it precisely or up to some symmetries inside. We can only really do this for very tiny models. Like, those big models, especially things which have softmaxes inside are very hard to invert. It's like yeah. It's a yeah. It's it's unclear how to steal models. Although some of us, Nicholas, for example, managed to steal parts of big models. But, yeah, I I don't think we know how to do this against the the frontier frontier. Well, what about this whole elephant in the room thing, you know, with China supposedly? Did they? Didn't they?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.