Dwarkesh Patel: evaluation

Transcript context

…All of these frictions on the political-type decisions that we are accustomed to only trusting humans for—legislation, being a judge, being a juror, or all the licensing that keeps certain professions human—that all strikes me as transitional. What we expect to come from a human and how we organize our politics has changed so many times throughout history, from little hunter-gatherer bands to empires to whatnot. Once an AI-run political system is much more efficient than the alternatives, those will probably tend to out-compete the others. Speaking of which, we’ve been talking about what preferences humans currently have and what impact that has on what kinds of goods will be scarce in the future. But of course, we’ll have different kinds of entities in the future: AIs. There was a time when there were no humans on Earth, but evolution selected for agents that have specific drives and preferences because those tend to survive the most, and those preferences now determine what a hundred-trillion-dollar world economy produces. Why not expect the same thing from AIs in the future? This is not even a world with catastrophic misalignment, where they just kill everybody. But there will be evolution of, even if not individual AIs, firms which have AIs as part of them. What will that evolution favor? It will probably favor firms or agents that grow. There’s a selection argument that things which grow will be more prevalent. Maybe just based on that, you can make some predictions about what their preferences will be. Is the kind of entity which prefers to have human-intrinsic goods going to be the kind of entity that accumulates resources the most? Probably not. Probably it saves more and has unsatisfiable demand for whatever the relevant resource happens to be. Compute is an obvious one. Can we use that to make some predictions about the non-human preferences that will be guiding the future? If there’s an AI that has its own welfare, is fully autonomous, and is making its own decisions that are welfare-relevant, to be honest, I have absolutely no prior that it would prefer to deal with humans. There’s no reason. But let me take the other side of that argument. Humans’ preferences to be interacting with one another, to trust and empathize with other humans versus a simulated AI, I think it’s a really important question whether those will change. I’ve heard a lot of arguments saying, “Look, right now we’re just not used to the technology. What you’re thinking of as relational… At some point, people are just going to see an AI therapist as a superior product, and they’re not going to need the empathy that the human is providing.” I think this is actually a really complicated question. Here’s one argument for why it’s not going to go away, and it has to do with evolution. Let’s say there are two types of people. One person doesn’t really have this preference. They can just interact with an AI, whatever can simulate it better. The other one has almost a moral emotion—using Jonathan Haidt’s framework—against offloading those sorts of social interactions to an AI. Which of those two people are going to reproduce, find a mate, all of these sorts of things? I think the answer is clear. It’s the second one that has the preference for other people.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.