Speaker unverified: belief

Transcript context

…No, it's, it's, it's about, it's about moral realism. I mean, Eliezer's frame of kind of what, what to expect of super intelligence to be motivated by is that there's some function of the state of the world that it wants to maximize the expected value of. And there's a lot of theory that, you know, the complete class theorems and the von Domin Morgan string theorems and the Dutch book theorems and all the rest that all suggests that any agent that isn't trying to maximize the expected value of some state of the world is going to get eaten. And so when I talked to Eliezer about this, which I haven't in many years, but when I did, he would say like, OK, so yeah, like maybe there will be like this coalition of weak sauce AIS, but like they're going to get eaten by the, the, the, the, the, you know, the actual strong AIS that are doing optimization. So yeah, I think there's there's some, you know what you could call a non scientific question, but from the evolutionary game theory point of view, it kind of is a scientific question. And from the a causal view, it's kind of a mathematical question, which is like, is there a dominant strategy for how to do well in the universe or in the multiverse? And I think there is a dominant strategy and it involves cosmopolitanism and pluralism and cooperation and, you know, mutual information and truth and harmony and all, you know, kind of all the, all the good things that human culture has discovered flow from this, this kind of this is the right strategy for how to be. And thus A sufficiently intelligent system would figure it out. And all of the all of the drama comes in the kind of adolescence of developing some capabilities ahead of others. Whereas for Eliezer, all of the, you know, the, the kind of alignment of what a system is trying to do is arbitrary. And humans have a particular kind of collection of values that for Eliezer, I think, I don't want to say this too strongly because I've got, you know, I haven't had this conversation, but I think he would say the reason that human values are worth installing. Is that they are our values and so we do better according to them by propagating them to our successors. And my point of view is like that's like, that's like looking at science and being like the thing that's good about human science is that it's our science. These are our beliefs. And so our successors will be more honest, you know, more knowledgeable about reality, if they, if they believe the true things according to us, like what we believe are true. And that this would, this is kind of absurd because this would go against the communion revolution. I think what's good about human values is that human, you know, human civilizations through, through cultural evolution and before that biological evolution, have settled on some equilibria that are in the right basin of attraction. values is that human, you know, human civilizations through, through cultural evolution and before that biological evolution, have settled on some equilibria that are in the right basin of attraction. Where under sufficient self reflection, like Eliezer talks about coherent extrapolated volition, we actually end up being correct about what the right way to exist is and what is a good life. Whereas for Eliezer it's more like there are millions and millions of possible coherent volitions. And you know, we're, we happen to be in one of them and we have to make sure we stay in that one 'cause that's the one that we care about by definition. So how much kind of moral progress or moral change do you expect?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.