Zvi Mowshowitz: preference

Transcript context

…I think that Anthropic... So their, their overall goal, right, or at least the goal as we understand it, is they are trying to be at the frontier of AI capability, and they are trying to pioneer ways to do this safely, right, for how they, what they feel is safe, while also, of course, making the money and creating the position to continue to be at the frontier and continue to make these improvements, and also people like money. And to eventually be able to build what they call powerful AI, which I generally call sufficiently advanced AI, is reasonably similar, such that we can then get all the nice things, but pre- create it in a way that we don't get all the terrible things, including potentially, you know, an existential risk or the, the end of human- the extinction of humanity. And also to help America and the world navigate this crucial time and ena- enact good policy and do the things that allow for the coordination necessary to ensure good outcomes and guard against bad outcomes. And, you know, they've been very consistently trying to wake the government up in various instances. They're trying to make them aware of how AI works, what the situation is, what AI can do, what it will be able to do, what risks this brings, and, uh, how to deal with those dangers. They've been relatively very conservative in what they call for the government to actually implement and do. They have-- they didn't get full-throated behind SB 1047, for example. They have not pushed for extremely aggressive regulations. They certainly have never pushed for anything remotely as aggressive as what just happened, even setting aside the fiasco level of implementation that was done, right? They are now calling for a, uh, licensing, de facto licensing regime. Now, the US government, in fact, has a de facto licensing regime. But what's going on right now, essentially, is they're just trying to deal with the implications of the model they've created and the fact that the US government is trying to deal with those implications while also not trusting or liking Anthropic very much, and also, like, pretty clearly not having any idea how any of this works, like, on a technical level, and not understanding what they're doing. So judging things based on vibes and political affiliation and associations and on who is willing to, like, respect their authoritah and bend the knee and, like, potentially give them various other things that they might want. And there's a huge communication and culture clash going on as part of this. But fundamentally speaking, what Anthropic is trying to do is give the public very powerful models and use those models in ways that enhance our safety and security rather than degrade it, even if they look really effing stupid while doing it with the classifiers and, and so on, because that's what they feel it takes to do this. at enhance our safety and security rather than degrade it, even if they look really effing stupid while doing it with the classifiers and, and so on, because that's what they feel it takes to do this. Uh, my guess is that the US government did not in any way feel it was necessary to put this level of control on biology and chemistry that they chose to do. I think that's them deciding this was necessary basically on their own. However, the US government clearly does care quite a bit about the controls on cyber. So I guess very high-level assessment first. What you said basically rings true to me. I think that's a good description as far as I understand of what it is they're trying to do. An additional wrinkle that I think you often hear from folks at Anthropic is, like, we need a leader that is going to be inclined to burn their lead at a critical time to use the advanced AIs that they and only they will have at that time to, like, solve all these safety and alignment problems in a super compressed, um, timeframe. And I've always been a little skeptical or allergic to that line of thinking because it certainly has a, you know, better us than them vibe to it, and I, I do worry that, that may be the, you know, the stuff with which the road to hell is paved. Are you, like, buying it at a high level? Like, are you sort of happy that they are racing ahead and leading and seemingly building some amount of lead over certainly everybody but maybe OpenAI, who, you know, is probably not too far behind on, on something similar? And do you think that, like, they will burn that lead when the time comes? Will they be allowed to burn the lead, and, and will it be productive? Like, macro strategy-wise, do you think this is a good strategy that you are happy they are playing?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.