Dwarkesh Patel: belief

Transcript context

…I do think succession to digital intelligence or augmented humans is inevitable. I have a four-part argument. Step one is, there’s no government or organization that gives humanity a unified point of view that dominates and that can arrange... There’s no consensus about how the world should be run. Number two, we will figure out how intelligence works. The researchers will figure it out eventually. Number three, we won’t stop just with human-level intelligence. We will reach superintelligence. Number four, it’s inevitable over time that the most intelligent things around would gain resources and power. Put all that together and it’s sort of inevitable. You’re going to have succession to AI or to AI-enabled, augmented humans. Those four things seem clear and sure to happen. But within that set of possibilities, there could be good outcomes as well as less good outcomes, bad outcomes. I’m just trying to be realistic about where we are and ask how we should feel about it. I agree with all four of those arguments and the implication. I also agree that succession contains a wide variety of possible futures. Curious to get more thoughts on that. I do encourage people to think positively about it. First of all, it’s something we humans have always tried to do for thousands of years, try to understand ourselves, trying to make ourselves think better, just understanding ourselves. This is a great success for science, humanities. We’re finding out what this essential part of humanness is, what it means to be intelligent. Then what I usually say is that this is all human-centric. But if we step aside from being a human and just take the point of view of the universe, this is I think a major stage in the universe, a major transition, a transition from replicators. We humans and animals, plants, we’re all replicators. That gives us some strengths and some limitations. We’re entering the age of design because our AIs are designed. Our physical objects are designed, our buildings are designed, our technology is designed. We’re designing AIs now, things that can be intelligent themselves and that are themselves capable of design. This is a key step in the world and in the universe. It’s the transition from the world in which most of the interesting things that are, are replicated. Replicated means you can make copies of them, but you don’t really understand them. Right now we can make more intelligent beings, more children, but we don’t really understand how intelligence works. Whereas we’re reaching now to having designed intelligence, intelligence that we do understand how it works. Therefore we can change it in different ways and at different speeds than otherwise. In our future, they may not be replicated at all. We may just design AIs, and those AIs will design other AIs, and everything will be done by design and construction rather than by replication. I mark this as one of the four great stages of the universe. First there’s dust, it ends with stars. Stars make planets. The planets can give rise to life. Now we’re giving rise to designed entities. I think we should be proud that we are giving rise to this great transition in the universe. It’s an interesting thing. Should we consider them part of humanity or different from humanity? It’s our choice. It’s our choice whether we should say, “Oh, they are our offspring and we should be proud of them and we should celebrate their achievements.”Or we could say, “Oh no, they’re not us and we should be horrified.” It’s interesting that it feels to me like a choice. Yet it’s such a strongly held thing that, how could it be a choice? I like these sort of contradictory implications of thought.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.