Holden Karnofsky: belief

Transcript context

…Gotcha. Now, it seems like the specific examples you used of energy and AI innovations are probably the hardest things to automate, given the fact that those are the ones that humanity's only gotten around to advancing most recently. Can you talk us through the intuition about how those might be easier? I think some of the stuff that might be hardest to automate would just be stuff that in some sense doesn't have anything to do with software or capabilities. So an example of something that might just be extremely hard to automate is trust, making a business deal, or providing care for someone who's sick. It might just be that even if an AI system has all the same intellectual capabilities as a human, and can write poetry just as well, have just as many ideas, and have just as good a conversation, it just doesn't look like a human. So people don't want that. Maybe you can create a perfect representation of a human on a screen, but it's still on the screen. In general, I see the progress in AI as being mostly on the software front, not the hardware front. So AIs are able to do a lot of incredible things with language, things with math, and things with board games. I also wouldn't be surprised if they could write hit music in the next decade or two. But these people really are not making the same kind of progress with robotics. So weirdly, a task that might be among the hardest to automate is the task of taking this bottle of water and taking off the cap. Because I have this hand that is just well designed for that. Well, it's clearly not designed for that, but it's like these hands can do a lot of stuff. We aren't seeing the same kind of progress there. So I think there are a lot of places where AI systems might have the kind of brains that can do roughly everything human brains can. There's some other reason they can't do some key economic tasks, but I think these are not the tasks I see likely to bottleneck the R&D as much. Gotcha.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.