Sander Schulhoff: prediction

Transcript context

…Okay. Maybe a final question. So at this point, hopefully this conversation raises people's awareness and fear levels and understanding of what could happen. So far nothing crazy has happened. I imagine as things start to break and this becomes a bigger problem, it'll become a bigger priority for people. If you had to just predict, say, over the next six months, year, couple years, how you think things will play out, what would be your prediction? When it comes to AI security, the AI security industry in particular, I think we're going to see a market correction in the next year, maybe in the next six months, where companies realize that these guardrails don't work. And we've seen a ton of big acquisitions on these companies where it's a classical cybersecurity companies like, "Hey, we got to get into the AI stuff," and they buy an AI security company for a lot of money. And I actually don't think these AI security companies, these guardrail companies are doing much revenue. I know that, in fact, from speaking to some of these folks. And I think the idea is like, "Hey, we got some initial revenue, look at what we're going to do." But I don't really see that playing out. And I don't know companies who are like, "Oh yeah, we're definitely buying AI guardrails. That's a top priority for us." And I guess part of it, maybe it's difficult to prioritize security or it's difficult to measure the results, and also companies are not deploying agentic systems that can be damaging that often, and that's the only time where you would really care about security. So I think there's going to be a big market correction in there where the revenue just completely dries up for these guardrails and automated red teaming companies. Oh, and the other thing to notice, there's just tons of these solutions out there for free, open source, and many of these solutions are better than the ones that are being deployed by the companies. So I think we'll see a market reaction there. I don't think we're going to see any significant progress in solving adversarial robustness in the next year. Again, this is something it's not a new problem, it's been around for many years, and there has not been all that much progress in solving it for many years. And I think very interestingly here, with image classifiers, there's a whole big ML robustness, adversarial robustness around image classifiers, people are like, "What if it classifies that stop sign as not a stop sign and stuff like that?" And it just never really ended up being a problem. Nobody went through the effort of placing tape on the stop sign in the exact way to trick the self-driving car into thinking it's not a stop sign. But what we're starting to see with LLM powered agents is that they can be tricked and we can immediately see the consequences, and there will be consequences. And so we're finally in a situation where the systems are powerful enough to cause real world harms. And I think we'll start to see those real world harms in the next year. Is there anything else that you think is important for people to hear before we wrap up? I'm going to skip the lightning round. This is a serious topic. We don't need to get into a whole list of random questions. Is there anything else that we haven't touched on? Anything else you want to just double down on before we wrap up?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.