Zvi Mowshowitz: evaluation

Transcript context

…u don't to worry about the person who moves faster. You don't have to worry about, like, the humans being forced to delegate to their to somebody else's AIs because, like, if I don't do it, they will. If the AI, like, has can have can have, like, guidelines as to what it will and will not do that are the same for everybody. And so, like, we can reserve areas of life, areas of action, areas of decision making. You know? Like, we can do all these things potentially, but at the cost of potential centralization, at the cost of concentration of power, the cost of somebody who's making those some some group of people is making those decisions about what this AI is going to do, or they're not, which is worse. Right? Like, in some important sense because I know what's making the decision. But if there's a lot of AIs, importantly, no one's making the decisions. And there's no good answers. It's the thing. Right? Nobody has a good solution to this. And that's why to circle it back to Davedot's comment, right, like to to unify it, this idea of if we solve the technical problem, we're 95% succeed, ignores the narrow path problems that we then have to walk anyway. Right? Where, like, if we don't have any ability to steer the situation and nobody is in control of it, then naturally, the AIs end up being put in charge of more and more things and things go to their natural competitive capitalist style endpoint where, like, we are fundamentally uncompetitive, and then we, like, get disempowered and then, like, things go badly. But if someone's in control and someone's in control with some methodology, what is it? Who? How? Right? Like, who is making those decisions? And then if an AI if we then, like, entrust a single AI or a single group of AI that also is a decision, many of decisions are good. Like, all of them sound terrifying. I think a lot of the times we're telling people are picking their poison. And either they're picking the poison they can live with or they're picking the poison they can't live with, and then acting accordingly. Right? They they're saying, like, well, you know, I don't trust concentration of power. Therefore, I'm gonna choose the solution that avoids concentration of power and then hope the rest works out kind of magically without a reason to expect it. And a lot people are speaking that because they understand conservation of power. They know conservation of power is real, but they don't really understand AGI or certainly superintelligence. I think those things kind of aren't real or hoping they're not real. And so, like, they're just hoping the other problem kinda doesn't come up even though they under even though, like, they kind of had this like, they don't even necessarily get to the point where they realize that, like, they have ruled out all possible solutions to this other problem if it does come up by going too far the other way. And so, again, I don't have an answer that anyone's gonna like for any of this. It's that you're choosing between bad choices and but I think I think we should just be cognizant of all these things. . And so, again, I don't have an answer that anyone's gonna like for any of this. It's that you're choosing between bad choices and but I think I think we should just be cognizant of all these things. Like, if if we do have to chase the future, if things are in fact going to be we're automating the r and D to our extent. I think that, like, the only thing you can do is say no. Don't do that. Don't develop these AIs that quickly that have this much capability. You need to slow your roll because we do not know how to handle this. Like, we we have all the different solutions, all of them are all the different ways to try and navigate that, and all of them are bad, both because we think the alignment problem is unsolved. It is that, like, we don't think the AI will be able to just solve it along the way moving as fast as possible. It's definitely unsolved now. But, like, we don't think that we will naturally get a good solution to that problem on various different levels, and you have to solve it on all these different meta levels at once. You can't just solve it on one level. Can solve it on the first try. And then, even if we did solve…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.