Flo Crivello: evaluation

Transcript context

…Can we unpack the threat model there, though? Because I'm a little bit like, okay. These are I don't know where you're running your ring full of inference, but, like, most American companies that are running Chinese models are not, like, calling the DeepSeek API. Right? They're using some Yep. American inference provider. Yep. So they can't, like, rug pull the model itself. They could have, like, sleeper agents in there. I think we're getting decent enough at through kind of JSPACE and various interpretability techniques that that's I wouldn't call that by any means a solved problem, but, like, from what I've seen in anthropic research, when they do the kind of one team with a sparse autoencoder versus one team without, like, these techniques are really allowing them to find these internal sleeper agent style problems in models with greater and greater efficiency and reliability. So I'm, like, optimistic that even if they were to train in some sort of 2027, you become an evil AI, that we'd be able to sniff that out and keep that to a manageable risk level. And then I also wonder about kind of a market mechanism, like, maybe insurance should instead of a ban, like, what about an insurance requirement? I think that would be maybe healthy for AI across the board, and then we could start till some of these risks. If Linde is powered entirely by Claude, maybe you get a cheaper rate on your insurance. If it's powered by DeepSeek and we there's some unknowns, like, maybe you have a higher rate on your insurance. Maybe that kind of levels the total cost out for you in a risk adjusted way. But it still, like, lets people take advantage of these global public goods that China is providing, which the rest of the world is not about to ban, obviously. Right? We would be doing this entirely to ourselves without any expectation that anybody else will follow suit. And we had a hard enough time getting people to, like, sign on to our Huawei ban. I think this like, the idea that people are gonna turn off deep sea entirely in Brazil or whatever, that's like a total nonstarter, I have to imagine. So what about, yeah, audits and internals and insurance? Like, can't we layer on a few things like that and get to a decent place? I would be down. I I I would be down. The the problem, though, is is it is a public good. And so who? Who's gonna do it? By the way, I I think, like, yes, we are making progress in mechanistic inter interoperability, but it is not it it is not yet a solved problem. Like, we don't know what lies in those models. And so, yes, there could be just like a backdoor of, like, you say, the magic wheel to the model, and all of a sudden, it does whatever you want. And only the CCP has this magic wheel. Even if it doesn't have that, you know, it's gonna have biases that are going to reflect CCP priorities. And, again, like, the the thing is just the most obvious example, but there there may be a lot more. Right? And we don't again, we don't know. And, yes, you could imagine retraining those models, but it's who's going to do it. And and why would they do it if there was no market demand for it? Right? It's not like people really care that much about the short term because it's it's a national interest thing. You know? Like, me as a as a private company, I'm like, do I really care? Like, all my users really asking about Canon Mill that often? Like, as as a business owner, I'm like, it's not directly aligned with my with my interest. But as a citizen, I'm immensely concerned. And so, yeah, I would be I would be in favor of a of a type of regulation that says, like, non Chinese models, like sorry. Non fine tuned and, like, sanitized change models are not welcome in The US. And then we would need I I am in favor of, like, an FAA for for AI. I I do think we need a new agency to regulate those models, and I think it would probably be the one that would be in charge of saying, okay. This model is for sure. We we fine tuned it enough, and and it's now representative of American interest. And probably we will have a sort suite of evals and whatnot to to verify that that's the case. I'd be I'd be I'd be I'd be open to that for sure. I think the strongest argument against this sort of ratcheting up of tensions is simply we might need to do a coordinated controlled call it a slowdown, don't call it a slowdown, but some sort of deliberate pacing of AI improvements. And we're gonna want China in on that deal. And I'm far from a China expert, but a couple of things I do feel pretty confident on coming coming back from two weeks, you know, knowing it all are like, one, if the state there agrees, I believe they can enforce on their companies whatever they agree to. So that's like I think they have that actually in much greater strength than we do. And two is if shit's going really crazy, it's gonna be in their own just sane self interest to do some deals with us because indeed we are ahead and they're I think there's like many deals that they would rationally take. And if it's in their rational self interest, then we can hopefully get mostly around a lot of the trust and defection problems. But I think we make it a lot harder for ourselves to get to those deals when we have all these aggressive postures toward them, of which banning their models would honestly affect them less than like our other one. They would affect them a lot less. What do they care if we ban their models, right, compared to refusing to sell them chips and refusing to sell them Claude? But it's just another log on the fire of we don't trust you. We can't deal with you. We're we assume you're a bad actor. And it seems like it makes it hard to get to the Yeah. Highest stakes agreements that we might really need.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.