Dwarkesh Patel: prediction

Transcript context

…Yeah, I definitely think that recent developments in AI have made me a bit more freaked out. Ever since I wrote The Most Important Century series and before that, there were years when Open Philanthropy was very interested in AI risk, but it's become more so as we've seen progress in AI. I think what we're seeing is we're seeing these very generic, simple systems that are able to do a lot of different tasks. I think people are interested in this. There are a lot of compilations of what GPT-3 is–– a very simple language model that, by the way, my wife and brother-in-law both worked on. This very simple language model just predicts the next word it's going to see in a stream of text. People have gotten it to tell stories. People got similar (though not identical) models to analyze and explain jokes. People have gotten it to play role-playing games, write poetry, write lyrics, answer multiple-choice questions, and answer trivia questions. One of the results that I found most ridiculous, strange and weird was this thing called Minerva, where people took one of these language models and with very little special intervention, they got it to do these difficult math problems and explain its reasoning and get them right about half the time. It wasn't really trained in a way that was very specialized for these math problems, so we just see AI systems having all these unpredictable human-like abilities just from having this very simple training procedure. That is something I find kind of wild and kind of scary. I don't know exactly where it's going or how fast. So if you think transformative AI might happen this century, what implications does that have for the traditional global health and well-being stuff that OpenPhilanthropy does? Will that have persistent effects of AI if it gets aligned? Will it create a utopia for us anyway? Holden Karnofsky I don't know about utopia. My general take is that anything could happen. I think my general take on this most important century stuff, and the reason it's so important is because it's easy to imagine a world that is really awesome and is free from scarcity and we see more of the progress we've seen over the last 200 years and we end up in a really great place. It's also easy to imagine a horrible dystopia. But my take is that the more likely you think all this is, the more likely you think transformative AI is, the more you should think that that should be the top priority, that we should be trying to make that go well instead of trying to solve more direct problems that are more short term. I'm not an extremist on this. So, OpenPhilanthropy does both. OpenPhilanthropy works on speculative far-off future risks and OpenPhil also does a bunch of more direct work. Again, we do direct and recommend a lot of money to give to those top charities, which do things like distributing bed nets in Africa to help prevent malaria and treat children for intestinal parasites. OpenPhilanthropy does a lot of advocacy for more money going to foreign aid or for better land use policies to have a stronger economy. We do a bunch of scientific research work that is more aimed at direct medical applications, especially in poor countries. So I support all that stuff. I'm glad we're doing it. It's just a matter of how real and how imminent do you think this transformative AI stuff is? The more real and more imminent, the more of our resources should go into it. Dwarkesh Patel Yeah, that makes sense to me. I'm curious, whatever work you do elsewhere, do those still have persistent effects after transformative AI comes? Or do you think they’ll basically wash out in comparison to the really big stuff? I mean, I think in some sense, the effects are permanent in the sense that if you cause someone to live a healthier, better life, that's a significant thing that happened. Nothing will ever erase that life or make that life unimportant, but I think in terms of the effects on the future, I do expect it mostly to wash out. I expect that mostly whatever we do to make the world better in that way will not persist in any kind of systematic, predictable manner past these crazy changes. I think that's probably how things look pre and post-industrial revolution. There are probably some exceptions, but that's my guess.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.