Andrej Karpathy: belief

Transcript context

…But I guess, less abstractly and more qualitatively, do you expect something to feel like… Because this thing can either think so fast, or has so many copies, or the copies can merge back into themselves, or is much smarter, any number of advantages an AI might have, will the civilization in which these AIs exist just feel qualitatively different from humans? I think it will. It is fundamentally automation, but it will be extremely foreign. It will look really strange. Like you mentioned, we can run all of this on a computer cluster and much faster. Some of the scenarios that I start to get nervous about when the world looks like that is this gradual loss of control and understanding of what’s happening. I think that’s the most likely outcome, that there will be a gradual loss of understanding. We’ll gradually layer all this stuff everywhere, and there will be fewer and fewer people who understand it. Then there will be a gradual loss of control and understanding of what’s happening. That to me seems the most likely outcome of how all this stuff will go down. Let me probe on that a bit. It’s not clear to me that loss of control and loss of understanding are the same things. A board of directors at TSMC, Intel—name a random company—they’re just prestigious 80-year-olds. They have very little understanding, and maybe they don’t practically actually have control. A better example is the President of the United States. The President has a lot of fucking power. I’m not trying to make a good statement about the current operant, or maybe I am, but the actual level of understanding is very different from the level of control.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.