Holden Karnofsky: belief

Transcript context

…Right, right. When I look throughout history, it often seems like people who predict long-term trends are too pessimistic. In the 70s, you might have been too pessimistic about the ability to find more oil or feed a growing population because you couldn't have predicted the technological breakthroughs that might have made these things possible. Does this inform some sort of vague optimism about the future for you with regards to AI or not? I think historically, people have been overly pessimistic about future technologies. I think by default, the picture with AI looks really scary. It just looks like it would be really easy to get a bad future in a lot of different ways if we just didn't move cautiously. These two considerations balance each other out a little bit for me. I know a lot of people who believe that we're in deep, deep, enormous trouble, and this outcome where you get AI with its own goals wiping humanity off the map is almost surely going to happen. I don't believe that and this is part of the reason I don't believe it. I actually think the situation looks very challenging, very scary by default, and I think we're tending to overestimate how bad and how dire things are. So they balance out a little bit for me. Okay, gotcha. In many of these cases, it seems like it would be impossible to see the positive scenario come about. For example, if you were forecasting population in the 70s, is there some reasonable method by which you would have predicted this was not going to lead to some massive famine that kills a billion people? Or would that have been your focus in the 70s if Open Philanthropy was a thing back then?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.