Roman Yampolskiy: belief

Transcript context

…So are you thinking about mass murder and destruction of human civilization or are you thinking of with squirrels, you put them in a zoo and they don’t really know they’re in a zoo? If we just look at the entire set of undesirable trajectories, majority of them are not going to be death. Most of them are going to be just things like brave new world where the squirrels are fed dopamine and they’re all doing some fun activity and the fire, the soul of humanity is lost because of the drug that’s fed to it, or literally in a zoo. We’re in a zoo, we’re doing our thing, we’re playing a game of Sims, and the actual players playing that game are AI systems. Those are all undesirable because the free will. The fire of human consciousness is dimmed through that process, but it’s not killing humans. So are you thinking about that or is the biggest concern literally the extinctions of humans? I think about a lot of things. So that is X-risk, existential risk, everyone’s dead. There is S-risk, suffering risks, where everyone wishes they were dead. We have also idea for I-risk, ikigai risks, where we lost our meaning. The systems can be more creative. They can do all the jobs. It’s not obvious what you have to contribute to a world where superintelligence exists. Of course, you can have all the variants you mentioned where we are safe, we’re kept alive, but we are not in control. We’re not deciding anything. We’re like animals in a zoo. There is, again, possibilities we can come up with as very smart humans and then possibilities, something a thousand times smarter can come up with for reasons we cannot comprehend. I would love to dig into each of those X-risk, S-risk, and I-risk. So can you linger on I-risk? What is that?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.