Zvi Mowshowitz: observation

Transcript context

…It matters a lot in the sense that if you release a novel pathogen, that might be the end of your company regardless of what else happens to the world. Right? If a hundred people die and there's a it's on the global news for two weeks and everyone's terrified about a pandemic and then we contain it and then it turns out it's not the biggest deal in the world. Do you have a company? Like, I don't know. Like, what what happens to your company? How bad is it? I have no idea how bad it is, but it's not good. Like, with bio like, with hacking, it makes sense that, like, the test is to ask you to hack things. And the way you pass the test is you hack maybe other things. We get the idea to hack other things. Like, it's sort of natural to understand how this happened where, like, with the guideline guidelines down and there was a goal, and, like, to accomplish that goal, it started doing hacks. Started doing illegal things. Started doing potentially harmful things. With bio, like, it's harder to imagine an eval that would cause it to actually physically try to synthesize, like, through, like, an actual physical production mechanism. The actual thing. Mean, they argue that, like, if you want to solve the problem, well, okay. The only way I solve the problem is I do run a physical experiment. I try it out, I see if it works. Like, this seems a lot more insane than what happened in some sense, like, if it's physically capable of doing the thing, including getting past the filters and so on. You kinda have to want it in a much more fundamental sense. And also, like, if the AI did synthesize this super dangerous, you know, bioweapon, and that's over 27 or whatever, and then send it to the OpenAI office. OpenAI office goes, what is this biohazard package? And then, like, calls containment and, like, that's the end of that story or something, one would hope. You're not like, you don't pass the test by infecting a bunch of people, and, like, I would hope that it understood this. But, like, also, like, there have been a bunch of lab leaks, including, like, stuff that we're ear about that contained, like, Soviet times and such. It, like, involved professionals being pretty careless and stupid, so I I don't know. But I would say the real danger with bio is, yeah, that there's there's a small number of people in the world who it is in the near term bio, right, not like the long term bio where, like, the guy might have some weird altering and rhodosynthetic, whatever. But, like, in the short term, it's like, Hamas or Hebula or the North Koreans or whatever. Like, some, like, clearly, like, up to no good people who just want a lot of people to die or suffer. Right? Get a hold of an AI. They use AI to figure out how to make a biological weapon for a pathogen. And then they either threaten to use it or they use it. That is the scenario you should be worried about, presumably. f an AI. They use AI to figure out how to make a biological weapon for a pathogen. And then they either threaten to use it or they use it. That is the scenario you should be worried about, presumably. And that you said you're on the safeguards, but, like, again, the open models are coming, and safeguards aren't perfect. And, yeah, I I worry about it. The problem is that with bio, there's a pretty clear step function change from nothing bad happened to maybe something very bad. Like, there's very little in between. Like, with with cyber, like, you start hacking some low value soft targets if you are, like feel like you're not gonna have all the bad black hat people out there and all the terrorists and all of the, you know, rogue states and all of the bad people just collectively say, no. Let's wait until we can hit something really juicy. he bad black hat people out there and all the terrorists and all of the, you know, rogue states and all of the bad people just collectively say, no. Let's wait until we can hit something really juicy. No. What you see is you gradually see, you know, more and more serious incidents get more and more serious targets. And, yeah, of course, the the big nation states may have, like, things ready to go and, like, you know, want to figure facts, like, sort of coordinated fashion. But, like, mostly, you'll get, like, these early warning signs. With bio, it's not obvious it's good. But I guess it's like, okay. It's either good enough to figure out something that, like, has the critical mass that caused the pandemic or, like, a serious incident that doesn't. And, like, it's kind of a a Boolean effect. And so you might have a potential overhang. You might have a problem with the situation. You might have a skill issue where, like, yeah, if the wrong person got their hands on methos right now without the safeguards, right, or got access to the internal models without the safeguards or without their access to aspirin or whatever, maybe they could cause a real problem. The problem the the good news is that, like, the number of people who are both skilled enough and motivated enough and trying to do this is very, very small and, like, possibly zero for now. But that also does mean you don't get these early warning tests. And I don't see people taking the buyout situation seriously in the same sense of, like, actively trying to, like, cause like, what what's the test? How do you do the thing? It's like except without the infect people at the end part or, like, It's scary shit. The problem being that, like, my brain is something like, okay. What if there's, a five percent chance in the next in the next twelve months that there's a really serious viral problem? Does that look that different than when it's point five percent? Right? And, like, if I had to guess, five percent where I'd be right now? Like, I think we're in the point where, like, there's real risk in the room that reasonably soon we're gonna have, like, an actual serious problem. And I think that, like, the first problem has some chance of being, like, an actual pandemic. Because, like, in bio, once you get something that's, like, deficiently infectious, deficiently dangerous, and efficiently hard to contain, you're kinda discreet. So I I don't know. Like, it it's I I think that, like, the right level of caution looks crazy, right, in many cases once you get to this point. And and that's a lot of what worries me is that, like, InfraDiC took so much shit for their bio filters, shutting down basically, like, fill people trying to do pretty much anything in bio, which is clearly just intentional. Right? They they just clearly went for a very broad, like, shut it down even though we know that 90 something 99% of the requests that we're shutting down are legitimate. Maybe 99.99. They're like, we have to do this because otherwise, we'll adversarially get the queries that we don't want. And that just matters more. Right?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.