Tim Scarfe: belief

Transcript context

…Mhmm. Interesting. I mean, they certainly could have a lot of sensors. If we're saying that there needs to be a lot of extra variety from elsewhere and it wouldn't be, you know, what's just inside of the data center, I think they could aggregate a lot of information and soak that up and process a lot more of that. So I still think they could have some type of advantage of at least doing this a lot faster than people. But certainly, they're in a vacuum only speaking by themselves or or or only only, working by itself, for instance, not learning thing. If there's too much correlation in that population, for instance, that might have the exploration budget and the the the yeah. I suppose mainly the exploration budget, be be too low and then it wouldn't have us there wouldn't be sufficient variety. I mean, evolution generally. I mean, there's there's, I think what was it? Fisher's fundamental theorem or something like that, which is, you know, the the rate of adaptation is in some ways directly proportional to the amount of variation and just sort of pointing at ways in which it's, lacking in variation. But I think that, some of that could be made up for, potentially. It could at least have the sensors that humans have, and more. Yeah. Another very interesting thing in your paper because when I think about AI risk in general, we we've thought about this a little bit on the show before, is in terms of, stability and destabilization and the relationship between offense and defense. And you use this term offense dominant, and you were saying that a destabilizing force would be like, if if the AI is offense dominant, then the defensive side of the equation couldn't catch up. Because at the moment, if you imagine our our kind of state of affairs, we have a kind of Nash equilibrium, right, where there are these countervailing factors on the on the offense and and the defense side. Can can you tell me about that? Yeah. So I think it I think the offense defense balance varies a lot by domain. Potentially, like, many information battles, it might actually be, like, for instance, like debates about the world might be a bit more defense dominant, which would be a reason for things like free speech. Meanwhile, other things might have more of a duality where an increase in the so for instance, really expert level, or really competent, computer security teams might experience more of an offense defense balance where something is identified, we patch the vulnerability very quickly, and so the attackers keep up with the defenders quite well. In other domains, like the software for critical infrastructure, there's more of an offense dominance or attacker's advantage because a lot of the software just doesn't get updated quickly. There's interoperability constraints. The software developer is no longer around. The software was made 30 plus years ago. Nobody even knows that it's there. It's things of things of that sort. There aren't, you know, specific there aren't strong enough economic incentives for doing this. There are uptime requirements. And so software and critical infrastructure for various forms of critical infrastructure is more of a sitting duck. In there, you don't experience the the a good offense defense balance. Likewise for bioweapons. Certainly, we have medicine, but we don't have cures for everything. In fact, we spend a lot trying to find cures for many sorts of diseases and ways of addressing certain viruses. So it's not necessarily a case that, oh, there's a new pathogen, and we'll just find a cure, you know, a a day later, and it will be mass proliferated across the globe, everything's taken care of. So there is a substantial delay, and so it is more offense dominant there too. And there are ways you can imagine the attacker having a really substantial advantage, like it propagating throughout this society before anybody showing symptoms and lacking various monitoring mechanisms. We were kind of sitting ducks for some of those. So it varies by domain. I think some parts of cyber, there's a good offense defense balance. Other parts of cyber, there isn't. Bio seems pretty offense dominant. And I think this affects how you want to whether you want to propagate the technology. If it's potentially catastrophic and offense dominant, that's something you want. Like, basically, if it's a WMD, like the WMD state, that's not something you wanna give to everybody. You don't wanna give everybody a nuke to make everybody safe. That's that's not how it works. It's it's people constraining each other's intent, some people just won't have their intent to be that constrainable. Meanwhile, other things, like defenses, like, home security systems or fences or whatever, you you'd want to propagate more. So so I I think that should affect the attitudes for specific types of AI capabilities as well of what's its offense dominance.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.