Ilya Sutskever: prediction

Transcript context

…It seems to me that this is a very precarious situation to be in. In the limit, we know that this should be possible. If you have something that is as good as a human at learning, but which can merge its brains—merge different instances in a way that humans can’t merge—already, this seems like a thing that should physically be possible. Humans are possible, digital computers are possible. You just need both of those combined to produce this thing. It also seems this kind of thing is extremely powerful. Economic growth is one way to put it. A Dyson sphere is a lot of economic growth. But another way to put it is that you will have, in potentially a very short period of time... You hire people at SSI, and in six months, they’re net productive, probably. A human learns really fast, and this thing is becoming smarter and smarter very fast. How do you think about making that go well? Why is SSI positioned to do that well? What is SSI’s plan there, is basically what I’m trying to ask. One of the ways in which my thinking has been changing is that I now place more importance on AI being deployed incrementally and in advance. One very difficult thing about AI is that we are talking about systems that don’t yet exist and it’s hard to imagine them. I think that one of the things that’s happening is that in practice, it’s very hard to feel the AGI. It’s very hard to feel the AGI. We can talk about it, but imagine having a conversation about how it is like to be old when you’re old and frail. You can have a conversation, you can try to imagine it, but it’s just hard, and you come back to reality where that’s not the case. I think that a lot of the issues around AGI and its future power stem from the fact that it’s very difficult to imagine. Future AI is going to be different. It’s going to be powerful. Indeed, the whole problem, what is the problem of AI and AGI? The whole problem is the power. The whole problem is the power. When the power is really big, what’s going to happen? One of the ways in which I’ve changed my mind over the past year—and that change of mind, I’ll hedge a little bit, may back-propagate into the plans of our company—is that if it’s hard to imagine, what do you do? You’ve got to be showing the thing. You’ve got to be showing the thing. I maintain that most people who work on AI also can’t imagine it because it’s too different from what people see on a day-to-day basis. I do maintain, here’s something which I predict will happen. This is a prediction. I maintain that as AI becomes more powerful, people will change their behaviors. We will see all kinds of unprecedented things which are not happening right now. I’ll give some examples. I think for better or worse, the frontier companies will play a very important role in what happens, as will the government. The kind of things that I think you’ll see, which you see the beginnings of, are companies that are fierce competitors starting to collaborate on AI safety. You may have seen OpenAI and Anthropic doing a first small step, but that did not exist. That’s something which I predicted in one of my talks about three years ago, that such a thing will happen. I also maintain that as AI continues to become more powerful, more visibly powerful, there will also be a desire from governments and the public to do something. I think this is a very important force, of showing the AI. That’s number one. Number two, okay, so the AI is being built. What needs to be done? One thing that I maintain that will happen is that right now, people who are working on AI, I maintain that the AI doesn’t feel powerful because of its mistakes. I do think that at some point the AI will start to feel powerful actually. I think when that happens, we will see a big change in the way all AI companies approach safety. They’ll become much more paranoid. I say this as a prediction that we will see happen. We’ll see if I’m right. when that happens, we will see a big change in the way all AI companies approach safety. They’ll become much more paranoid. I say this as a prediction that we will see happen. We’ll see if I’m right. But I think this is something that will happen because they will see the AI becoming more powerful. Everything that’s happening right now, I maintain, is because people look at today’s AI and it’s hard to imagine the future AI. There is a third thing which needs to happen. I’m talking about it in broader terms, not just from the perspective of SSI because you asked me about our company. The question is, what should the companies aspire to build? What should they aspire to build? There has been one big idea that everyone has been locked into, which is the self-improving AI. Why did it happen? Because there are fewer ideas than companies. But I maintain that there is something that’s better to build, and I think that everyone will want that. It’s the AI that’s robustly aligned to care about sentient life specifically. I think in particular, there’s a case to be made that it will be easier to build an AI that cares about sentient life than an AI that cares about human life alone, because the AI itself will be sentient. And if you think about things like mirror neurons and human empathy for animals, which you might argue it’s not big enough, but it exists. I think it’s an emergent property from the fact that we model others with the same circuit that we use to model ourselves, because that’s the most efficient thing to do.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.