Brad Carson: prediction

Transcript context

…Something tragic happens. Some people are killed. Some innocent innocent people are killed in war, some children, you know. And now that AI is involved, what happens with AI there? I think neural nets have changed the game in terms of the law of war. And, yeah, not just making it more complex, they make it completely opaque. And that's a problem. So that's not something that I accept. That's the problem with autonomy in war is that, you're right, people make mistakes. Right? We have killed innocents since the beginning of war. But there is a way to think of this problem, and we hold people accountable. We have after action. We say, like, how did we get this wrong? You know, who made the mistake here? What what what error that we can correct in the future can can we identify? You know, we know it's well known, right, that these kind of neural net systems are opaque to that. We don't know how they make decisions. And yes, that's a serious problem, and that's 1 of the main reasons that autonomy and war should be viewed enormously skeptically. Yeah. And let's circle back to autonomy in a minute, but this point about the opaqueness, or as my as my cohost talks about a lot, you know, the the not being intelligible. You just you can't And even though, sure, we've had people on the show to talk about interpretability, and there's the mech interpret people and all this. I made the comment 1 time maybe in 1 of the first interpretability shows we had. I was like, you know, something seems off. We're using mathematical models to explain mathematical models and I'm not sure we're really we're really making progress there. So it's extremely hard if it's even if it's even possible. So you have these black boxes that like you say, are not intelligible. They have very different failure modes from from human human failure modes. Right? I mean, not saying humans don't fail. I'm just saying the failure modes are extremely different. Like you've I think you had some examples maybe in 1 of your articles, like the the stop sign that you put like a little water bottle picture on it and suddenly it's not recognized as a stop sign anymore. Like that sort of thing. What I mean,…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.