Sander Schulhoff: evaluation

Transcript context

…Yeah. [inaudible 01:15:31] also also be quite bad. So that seems to be a potential direction, maybe a promising direction. I think another thing worth pointing out is looking at anthropic constitutional classifiers and other models, it does seem to be more difficult to elicit CBRN and other really harmful outputs from chatbots, but solving indirect prompt injection, which is basically prompt injection against agents done by external people on the internet is still very, very, very unsolved, and it's much more difficult to solve this problem than it is to stop CBRN elicitation, because with that kind of information, as one of my advisors just noted, it's easier to tell the model, "Never do this," than with emails and stuff, "Sometimes do this. " So with CBRN instead you can be like, "Never, ever talk about how to build a bomb, how to build atomic weapon. Never." But with sending an email, you have to be like, "Hey, definitely help out send emails, oh, but unless there's something weird going on, then don't send email." So for those actions, it's much harder to describe and train the AI on the line, the line not to cross and how to not be tricked. So it's a much more difficult problem. And I think adversarial training deeper in this stack is somewhat promising. I think new architectures are perhaps more promising. There's also an idea that as AI capabilities improve, adversarial robustness will just improve as a result of that. And I don't think we've really seen that so far. If you look at the static benchmarking, you can see that, but if you look at it still takes humans under an hour, it's not like you need nation state resources to trick these models. Anyone can still do it. And from that perspective, we haven't made too much progress in robustifying these models. Well, I think what's really interesting is your point that Anthropic and Claude are the best at this, I think that alone is really interesting that there's progress to be made. Is there anyone else that's doing this well that you want to shout out just like, "Okay, there's good stuff happening here," either a company, AI company or other models?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.