Holden Karnofsky: preference

Transcript context

…You've advocated for the moral parliament's approach when you're trying to make decisions. What is the right level of organization at which to use that approach? Should individuals be making decisions based on having multiple different moral parties inside them? Is that the right approach for entire movements but individuals should be specializing? What is the right level to be applying this approach at? Moral uncertainty is something I hope to write about in the future. The basic idea is that there might be a bunch of different ways about thinking about what the right thing is to do in the world. You might look at the world from one angle and say, “Well, what matters is like the total sum of all the pleasures. So therefore a bigger world would be better. So therefore I should be like really obsessed with getting the world to be as big as possible.” There might be another perspective that says that what really matters is suffering. “We should minimize suffering. We should want the world to be small.” There might be another perspective that says it doesn't matter what happens to the world. “It matters how I act. What matters is that I act with integrity, that I tell the truth, things like that.” There's these interesting debates asking what should you do when you think you have some sympathy for all these views. How do you choose an action that some perspectives would say is the best thing you've ever done and some would say is the worst thing you've ever done? The moral parliament idea is an idea that was laid out by Nick Bostrom in an Overcoming Bias post a decade ago that I like. I think about it as if I'm just multiple people. I just think about it as if there's multiple people all living inside my head arguing about what to do and they all are friends and they all care about each other and they want to get along. So they're trying to reach a deal that all of them can feel fairly good about. That is how I tend to think about dealing with different moral views. I tend to want to do things that are really good according to one and not too bad according to the rest and try to have the kind of different parts of myself making deals with each other. So that relates to something I said at the beginning about not being into ends justify the means. I put a lot of effort into doing things that would be like really, really good. If this most important century stuff came out true, that would be good but it also would not be too catastrophic if it didn't. So there are lines I'm not willing to cross. There are behaviors I'm not willing to engage in to promote the kind of goals of people who worry about AI safety. So it's a moderating approach I think. It makes a lot of sense for somebody who is the CEO of Open Philanthropy to want the decisions you make to reflect uncertainties about your decisions. However, if it's just somebody like me where I'm not in some sort of leadership position where I have a large amount of resources to dedicate to, should I just specialize in that particular moral view I have or should I also be trying to allocate my time and resources according to different moral views?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.