Holden Karnofsky: commitment

Transcript context

…Gotcha. Okay, so we'll get to forecasting in a second, but let's continue on the object-level conversation about the most important century. I want to make sure I have the thesis right. Is the argument that because we're living in a weird time, we shouldn't be surprised if something transformative happens in a century or is the argument that something transformative could happen this century? It's a weird time. So something we haven't covered yet, but I think is worth throwing in is that a significant part of the ‘Most Important Century series’ is making the case that even if you ignore AI, there's a lot of things that are very strange about the time that our generation lives in. The reason I spent so much effort on this is because back in 2014, my number one objection to these stories about transformative AI wasn’t anything about whether the specific claims about AI or economic models or alignment research made sense. This whole thing sounded crazy and was just suspicious. It's suspicious if someone says to you, “You know, this could be the most important century of all time for humanity.” I titled the series that way because I wanted people to know that I was saying something crazy and that I should have to defend it. I didn't want to be backpedaling or soft-pedaling or hiding what a big claim I was making. I think my biggest source of skepticism was how I didn’t have any specific objection. It sounds crazy and suspicious to say that we might live in one of the most significant times of the most significant time for humanity ever. So a lot of my series is saying that it is weird to think that, but we already have a lot of evidence that we live in an extraordinarily weird time that would be on the short list of contenders for the most important time ever–– even before you get into anything about AI, and just used completely commonly accepted facts about the world. For example, if you chart the history of economic growth, you’ll see that the last couple hundred years have seen faster growth by a lot than anything else in the history of humanity or the world. If you chart anything about scientific and technological developments, you’ll see that everything significant is packed together in the recent past. There's almost no way to cut it. I've looked at many different cuts of this. There's almost no way to cut it that won't give you that conclusion. One way to put it is that the universe is something like 11 or 12 billion years old. Life on Earth is three billion years old. Human civilization is a blink of an eye compared to that. We're in this really tiny sliver of time, the couple hundred years when we've seen a huge amount of technological advancement and economic growth. So that's weird. I also talk about the fact that the current rate of economic growth seems high enough that we can't keep it going for that much longer. If it went for another 10,000 years, that's another blink of an eye and galactic time scales. It looks to me like we would run out of atoms in the galaxy and wouldn't have anywhere to go. So I think there are a lot of signs that we just live in a really strange time. One more thing that I'll just throw in there–– I think a lot of people who disagree with my take would say, “Look, I do believe eventually we will develop space colonization abilities. y strange time. One more thing that I'll just throw in there–– I think a lot of people who disagree with my take would say, “Look, I do believe eventually we will develop space colonization abilities. We could go to the stars, fill up the galaxy with life, and maybe have artificial general intelligence, but to say that this will happen in a century is crazy. I think it might be 500 years. I think it might be a thousand years. I think it might be 5000 years.” A big point I make in the series is how I say, “Well, even if it's 100000 years, that's still an extremely crazy time to be in in the scheme of things.” If you make a graphic timeline and you show my view versus yours, they look exactly the same down to the pixel. So there are already a lot of reasons to think we live in a very weird time. We're on this planet where there's no other sign of life anywhere in the galaxy. We believe that we could fill up the galaxy with life. That alone would make us among the earliest life that has ever existed in the galaxy–– a tiny fraction of it. So that’s a lot of what the series is about. I'll answer this question explicitly. You ask, “Is this series about whether transformative AI will come and make this century weird?” or is it about “This century could be weird, and therefore transformative AI will come?” The central claim is that transformative AI could be developed in this century and the sections about ‘how weird a time we live in’ are just a response to an objection. It's a response to a point of skepticism. It's a way of saying there are already a lot of reasons to think we live in a very weird time. So actually, this thing about AI is only a moderate quantitative update, not a complete revolution in the way you're thinking about things.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.