Phil Trammell: prediction

Transcript context

…You’re also paying for a lot of regulation-type stuff. With lawyers particularly, you need some entity to back up the product. You need ownership of the product. You need somebody to be able to fire or hire, and there are licensing issues. There’s a lot of regulatory layers that are also going to be keeping—even if there’s no relational element—humans in the loop that have nothing to do with the ability of the human to actually perform the service. All of these frictions on the political-type decisions that we are accustomed to only trusting humans for—legislation, being a judge, being a juror, or all the licensing that keeps certain professions human—that all strikes me as transitional. What we expect to come from a human and how we organize our politics has changed so many times throughout history, from little hunter-gatherer bands to empires to whatnot. Once an AI-run political system is much more efficient than the alternatives, those will probably tend to out-compete the others. Speaking of which, we’ve been talking about what preferences humans currently have and what impact that has on what kinds of goods will be scarce in the future. But of course, we’ll have different kinds of entities in the future: AIs. There was a time when there were no humans on Earth, but evolution selected for agents that have specific drives and preferences because those tend to survive the most, and those preferences now determine what a hundred-trillion-dollar world economy produces. Why not expect the same thing from AIs in the future? This is not even a world with catastrophic misalignment, where they just kill everybody. But there will be evolution of, even if not individual AIs, firms which have AIs as part of them. What will that evolution favor? It will probably favor firms or agents that grow. There’s a selection argument that things which grow will be more prevalent. Maybe just based on that, you can make some predictions about what their preferences will be. Is the kind of entity which prefers to have human-intrinsic goods going to be the kind of entity that accumulates resources the most? Probably not. Probably it saves more and has unsatisfiable demand for whatever the relevant resource happens to be. Compute is an obvious one. Can we use that to make some predictions about the non-human preferences that will be guiding the future?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.