Holden Karnofsky: belief

Transcript context

…I'm glad you brought up the Industrial Revolution because I feel like there are two implicit claims within the most important century thesis that don't seem perfectly compatible. One is that we live in an extremely wild time and that the transition here is potentially wilder than any other transition there has been before. The second is we have some sense of what we can be doing to make sure this transition goes well. Do you think that somebody at the beginning of the Industrial Revolution, knowing what they knew then, could have done something significant to make sure that it went as favorably as possible? Or do you think that that's a bad analogy for some reason? It's a pretty good analogy for being thought-provoking and for thinking, “Gosh, if you had seen the Industrial Revolution coming in advance and this is when economic growth really reached a new level back in the 1700s and 1800s, what could you have done?” I think part of the answer is it's not that clear and I think that is a bit of an argument we shouldn’t get too carried away today by thinking that we know exactly what we can do. But I don't think the answer is quite nothing. I have a goofy cold-taste post that I never published and may never publish because I lost track of it. What it basically says is “What if you'd been in that time and you had known the Industrial Revolution was coming or you had thought it might be?” You would ask yourself what you could be doing. One answer you might have given is you might have said, “Well, gosh, if this happens, whatever country it happens in might be disproportionately influential. What would be great is if I could help transform the thinking and the culture in that country to have a better handle on human rights and more value on human rights and individual liberties and a lot of other stuff–– and gosh, it kind of looks like people were doing that and it looks like it worked out.” So this is the Enlightenment. I even give this goofy example–– I could look it up and it's all kind of a trollish post. But the example is someone's thinking, “Hey, I'm thinking about this esoteric question about what a government owes to its citizens” or, “When does a citizen have a right to overthrow a government or when is it acceptable to enforce certain beliefs and not?” Then the other person in the dialogue is just like, “This is the weirdest, most esoteric question. Why does this matter? Why aren't you helping poor people?” But these are the questions that the Enlightenment thinkers were thinking about. I think there is a good case that they came up with a lot of stuff that really shaped the whole world since then because of the fact that the UK became so influential and really laid the groundwork for a lot of stuff about the rights of the governed, free speech, individual rights, and human rights. Then I go to the next analogy. It’s like we're sitting here today and someone is saying, “Well, instead of working on global poverty, I'm studying this esoteric question about how you get an AI system to do what you want it to do instead of doing its own thing. I think it's not completely crazy to see them as analogous.” Now, I don't think this is what the Enlightenment thinkers are actually doing. I don't think they were saying this could be the most important millennium, but it is interesting that it doesn't look like there was nothing to be had there. It doesn't look like there's nothing you could have come up with. In many ways, it looks like what the Enlightenment thinkers were up to had the same esoteric, strange, overly cerebral feel at the time and ended up mattering a huge amount. u could have come up with. In many ways, it looks like what the Enlightenment thinkers were up to had the same esoteric, strange, overly cerebral feel at the time and ended up mattering a huge amount. So it doesn't feel like there's zero precedent either.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.