Nathan Labenz: prediction

Transcript context

…Yes. OK, so they're moral patients, but they're also because of their fundamental constitution, not in the sense of the written document, but the way that they are. They are, they are beings that are meant to be helpful, right? So there's, as I'm sure you're well aware, there's this line of thinking that's even if we get the alignment right, we might end up in a spot where we're quite unhappy because we'll hand over more and more responsibility and key decision making and ultimately kind of power to AIS because they're better at a lot of things. And then we'll end up disempowered. And that could happen gradually and gradual disempowerment. But if we but if it happens, we might end up in a spot where we realize we've lost control and now we're the animals in a zoo of our own construction now, like supervised hopefully. Well, we're not we can't we lost control over exactly what happens from there. So the so the analysis goes, do you think that this is a real worry or do you feel like the inherent tool, nature or desire to serve of the AIS like gets us out of that somehow? That's a really interesting framing. I I think gradual disempowerment of biological humans is 100% inevitable. And that has been a feature of my worldview for as long as I can remember, you know, as far back as as the age of spiritual machines that laid out a pretty clear story of where the trajectory is going. And, you know, we're talking about 100 years from now. Yeah. Biological humans are not gonna have any power even in aggregate. That's just. That's the way it is, I think. Not necessarily bad. I don't think having power is constitutive of flourishing for humans. I think this is a mindset shift that can be addressed through, you know, education and, like, therapy. Like you. You shouldn't need to be in charge of the universe to feel like you're getting a good shake. But yeah, I do think this is inevitable. And I think this is kind of because in, you know, whispering earring fashion, like, yeah, the best way to be in service does involve taking away gradually a lot of decision making power voluntarily because it's actually just better for everyone like this. That's that's the right way for things to go. And I think that will happen.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.