Holden Karnofsky: belief

Transcript context

…Maybe I'm a bit more pessimistic about that because I think the people who are working on individual rights frameworks weren’t anticipating an industrial revolution. I feel like the type of person who’d actually anticipate the industrial revolution would have a political philosophy that was actually probably a negative given, you know… Karl Marx. If you saw something like this happening, I don’t think it would be totally not obvious. I mean, I think my basic position here is that I'm not sitting here highly confident. I'm not saying there's tons of precedent and we know exactly what to do. That's not what I believe. I believe we should be giving it a shot. I think we should be trying and I don't think we should be totally defeatist and say, “Well, it's so obvious that there's never anything you could have come up with throughout history and humans have been helpless to predict the future.” I don't think that is true. I think that's enough of an example to kind of illustrate that. I mean, gosh, you could make the same statement today and say, “Look, doing research on how to get AI systems to behave as intended is a perfectly fine thing to do at any period in time.” It's not like a bad thing to do. I think John Locke was doing his stuff because he felt it was a good thing to do at any period in time, but the thing is that if we are at this crucial period of time, it becomes an even better thing to do and it becomes magnified to the point where it could be more important than other things. The one reason I might be skeptical of this theory is that I could say, “Oh, gosh, if you look throughout history, people were often convinced that they were living in the most important time,” or at least an especially important time. If you go back, everybody could be right about living in the most important time. Should you just have a very low prior that anybody is right about this kind of thing? How do you respond to that kind of logic?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.