Dwarkesh Patel: prediction

Transcript context

…Well, I thought I just gave an example. Suppose we don't know what kind of environment the ancestors of the domestic cats lived in. If it contains undergrowth, then dealing with undergrowth requires some very sophisticated programs. Otherwise, you will just get stuck somewhere and starve to death. If a dog gets stuck in a bush, it has no program to get out other than to shake itself until it gets out. It doesn't have a concept of doing something that temporarily makes matters worse and allows you to get out. Dogs can't do that. It's not because that's a particularly complicated thing. It's just that programming doesn't have that. But an animal's programming could easily have that if it lived in an environment in which that happened a lot. Is your theory of AI compatible with AIs that have narrow objective functions, but functions that, if fulfilled, would give the creator a lot of power? For example, if I wrote a deep learning program, I trained it over financial history and asked it to make me a trillion dollars on the stock market. Do you think that this would be impossible? Or if you think this would be possible, it doesn’t seem like an AGI, but it seems like a very powerful AI, right? So, it seems like AI is getting somewhere. Well, if you want to be powerful, you might do better inventing a weapon or something. But or a better mousetrap because it's nonviolent. You can invent a paperclip. To use an example was often used in this context. If paperclips weren't invented, you could invent a paperclip and make a fortune. That's an idea, but it's not an AI because it's not the paperclip that's going out there. It's your idea in the first place, which caused the whole value of the paperclip. Similarly, suppose you invent a dumb arbitrage machine that seeks out complicated trades to make that are more complicated than anyone else is trying to do. Suppose that makes you a fortune. Well, the thing that made you a fortune was not the arbitrage machine. It was your idea of how to search for arbitrage opportunities that no one else sees. Right? That is what was valuable. That's the usual way of making money in the economy—you have an idea, and then you implement it. AI is beside the point. It could have been the paperclip.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.