Speaker unverified: belief

Transcript context

…Another question from this side. As AI gets better — you were talking about in chess, I think. You said biology becomes a liability as AI gets better at thinking than we are. So what becomes the most important work of humans? Love, creativity, how we treat each other. We’re going to find out. I don’t know. I was talking to someone recently about all the different ways this can go. There are versions — a lot of people, a lot of techno-utopianists love them — where we all just merge with AI, and there’s no opt-out. Either you merge with AI or you become irrelevant. That sounds cool, probably, to a lot of people in this room and terrifying to 99 percent of people on Earth. We talked about the need for positive stories. I don’t think that’s the story that most people want to hear. I don’t think that’s exciting. I don’t think that’s a future that most people are excited about. But the thing that is really interesting about this is, sure, we’re going to build this thing that’s smarter than humans in every way, but we get to make a lot of decisions about how to build it. You only get to make them once, when the von Neumann probes launch from Earth, and eventually you never get to communicate again because of the speed of light. You better have gotten it right, but we do get to make them. I don’t know what the answer is. I do know I should not get to make it by myself. But maybe we as humans, as a whole, decide the deal we want with AI, like humans get Earth and a 100-light-year radius, and AI gets the rest of the universe, but we get to live our lives as we want, and maybe that’s OK. The option space is big, but we won’t be smarter.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.