Joscha Bach: evaluation

Transcript context

…But isn’t this everything? This love you have for other humans, you can call it responsibility, but that connection, that’s the ego death, isn’t that the thing we’re really afraid of, is not to just die, but to let go of the experience of love with other humans? This is not everything. Everything is everything, right? So there’s so much more and you could be lots of other things. You could identify with lots of other things. You could be identifying with being Gaia, some kind of planetary control agent that emerges over all the activity of life on earth. You could be identifying with some hyper Gaia that is the concatenation of Gaia or the digital life and digital minds. And so, in this sense, there will be agents in all sorts of substrates and directions that all have their own goals, and when they’re not sustainable, then these agents will cease to exist. Or when the agent feels that it’s done with its own mission, it’ll cease to exist. In the same way when you conclude a thought, the thought is going to wrap up and gives control over to other thoughts in your own mind. So, there is no single thing that you need to do, but I observe myself as a being, that sometimes I’m a parent and then I have an identification and a job as a parent, and sometimes I am an agent of consciousness on earth, and then from this perspective, there’s other stuff that is important. So, this is my main issue with Eliezer’s perspective, that he’s basically marrying himself to a very narrow human aesthetic. And that narrow human aesthetic is a temporary thing. Humanity is a temporary species, like most of the species on this planet are only around for a while, and then they get replaced by other species in a similar way as our own physical organism is around here for a while and then gets replaced by a next generation of human beings that are adapted to changing life circumstances and average via mutation and selection. And it’s only when we have AI and become completely software that we can become infinitely adaptable and we don’t have this generational and species change anymore. So, if you take this larger perspective and you realize it’s really not about us, it’s not about Eliezer or humanity, but it’s about life on earth or it’s about defeating entropy for as long as we can while being as interesting as we can, then the perspective changes dramatically and preventing AI from this perspective looks like a very big sin. But when we look at the set of trajectories that such an AI would take that supersedes humans, I think Eliezer is worried about ones that not just kill all humans, but also have some kind of maybe objectively undesirable consequence for life on earth. Like how many trajectories, when you look at the big picture of life on earth, would you be happy with, and how much worry you with AGI, whether it kills humans or not?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.