Sander Schulhoff: belief

Transcript context

…I love that. Okay. Maybe a last topic for say foundational model companies that are listening to this and just like, "Okay, I see, maybe I should be paying more attention to this." I imagine they very much are, clearly still a problem. Is there anything they can do? Is there anything that these LLMs can do to... ... Problem. Is there anything they can do? Is there anything that these LLMs can do to reduce the risks here? This is something I thought about a lot and I've been talking to a lot of experts in AI security recently, and I'm something of an expert in attacking, but wouldn't really call myself an expert in defending, especially not at a model level. But I'm happy to criticize. And so in my professional opinion there's been no meaningful progress made towards solving adversarial robustness, prompt injection jailbreaking in the last couple of years since the problem was discovered. And we're often seeing new techniques come out, maybe there are new guardrails, types of guardrails, maybe new training paradigms, but it's not that much harder to do prompt injection jailbreaking still. That being said, if you look at Anthropic's constitutional classifiers, it's much more difficult to get CBRN information out of Claude models than it used to be, but humans can still do it in, I'd say, under an hour, and automated systems can still do it. And even the way that they report their adversarial robustness still relies a lot on static evaluations where they say, "Hey, we have this data set of malicious prompts, which were usually constructed to attack a particular earlier model." And then they're like, "Hey, we're going to apply them to our new model." And it's just not a fair comparison because they weren't made for that newer model. So the way companies report their adversarial robustness is evolving and hopefully will improve to include more human evals. Anthropic is definitely doing this, OpenAI is doing this, other companies are doing this, but I think they need to focus on adaptive evaluations rather than static datasets, which are really quite useless. There's also some ideas that I've had and spoken with different experts about, which focus on training mechanisms. There are theoretically ways to train the eyes to be smarter, to be more adversarially robust, and we haven't really seen this yet, but there's this idea that if you start doing adversarial training in pre-training earlier in the training stack, so when the AI is a very, very small baby, you're being adversarial towards it and training it then, then it's more robust, but I think we haven't seen the resources really deployed to do that. What I'm imagining in there is an orphan just having a really hard life and just they grew up really tough, they have such street smarts, and they're not going to let you get away with telling you how to build a bomb. That's so funny how it's such a metaphor for humans in a way.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.