01 / belief
I fully agree with that. I think there’s an even stronger counterargument to this theory.
“I fully agree with that. I think there’s an even stronger counterargument to this theory.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
Dwarkesh Podcast / episode intelligence
Ilya Sutskever — We're moving from the age of scaling to the age of research
Speakers in the public record
Claim mix
belief 15evaluation 6prediction 4commitment 2uncertainty 2observation 1preference 1
Evidence policy
Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.
Claim ledger
The useful parts, with receipts.
31 published records
02 / belief
I think there are some similarities between both of these and pre-training, and pre-training tries to play the role of both of these.
“I think there are some similarities between both of these and pre-training, and pre-training tries to play the role of both of these.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
03 / belief
How do we elicit meaningful diversity among AIs? I think just raising the temperature just results in gibberish.
“How do we elicit meaningful diversity among AIs? I think just raising the temperature just results in gibberish.”
- Speaker
- Dwarkesh Patel
- Publisher
- Dwarkesh Podcast
04 / belief
I think that in our case, we have sufficient compute to prove, to convince ourselves and anyone else, that what we are doing is correct.
“I think that in our case, we have sufficient compute to prove, to convince ourselves and anyone else, that what we are doing is correct.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
05 / commitment
If the ideas turn out to be correct—these ideas that we discussed around understanding generalization—then I think we will have something worthy.
“If the ideas turn out to be correct—these ideas that we discussed around understanding generalization—then I think we will have something worthy.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
06 / belief
I think there’s a lot of merit because it’s very nice to not be affected by the day-to-day market competition.
“I think there’s a lot of merit because it’s very nice to not be affected by the day-to-day market competition.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
07 / belief
For things like vision, hearing, and locomotion, I think there’s a pretty strong case that evolution has given us a lot.
“For things like vision, hearing, and locomotion, I think there’s a pretty strong case that evolution has given us a lot.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
08 / evaluation
What I was alluding to with the person whose emotional center got damaged, it’s more that maybe what it suggests is that the value function of humans is modulated by emotions in some important way that’s hardcoded by evolution. And maybe that is important for people to be effective in the world.
“What I was alluding to with the person whose emotional center got damaged, it’s more that maybe what it suggests is that the value function of humans is modulated by emotions in some important way that’s hardcoded by evolution. And maybe that is important for people to be effective in the world.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
09 / belief
When do you expect that impact? I think the models seem smarter than their economic impact would imply.
“When do you expect that impact? I think the models seem smarter than their economic impact would imply.”
- Speaker
- Dwarkesh Patel
- Publisher
- Dwarkesh Podcast
10 / belief
There’ll be very strong economic forces for this, and I think the impact is going to be felt very strongly.
“There’ll be very strong economic forces for this, and I think the impact is going to be felt very strongly.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
11 / commitment
A lot of people’s models of recursive self-improvement literally, explicitly state we will have a million Ilyas in a server that are coming up with different ideas, and this will lead to a superintelligence emerging very fast.
“A lot of people’s models of recursive self-improvement literally, explicitly state we will have a million Ilyas in a server that are coming up with different ideas, and this will lead to a superintelligence emerging very fast.”
- Speaker
- Dwarkesh Patel
- Publisher
- Dwarkesh Podcast
12 / belief
What’s the good world? It’s where we have these powerful human-like learners that are also… By the way, maybe there’s another thing we haven’t discussed on the spec of the superintelligent AI that I think is worth considering.
“What’s the good world? It’s where we have these powerful human-like learners that are also… By the way, maybe there’s another thing we haven’t discussed on the spec of the superintelligent AI that I think is worth considering.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
13 / belief
I want to emphasize that I think the value function is something that’s going to make RL more efficient, and I think that makes a difference.
“I want to emphasize that I think the value function is something that’s going to make RL more efficient, and I think that makes a difference.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
14 / belief
Basically I think that there is a big benefit from AI being in the public, and that would be a reason for us to not be quite straight shot.
“Basically I think that there is a big benefit from AI being in the public, and that would be a reason for us to not be quite straight shot.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
15 / belief
I think eventual convergence on the technical approach is probably going to happen as well, but I was alluding to convergence to the alignment strategies.
“I think eventual convergence on the technical approach is probably going to happen as well, but I was alluding to convergence to the alignment strategies.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
16 / belief
There may be another blocker though, which is that there is a possibility that the human neurons do more compute than we think.
“There may be another blocker though, which is that there is a possibility that the human neurons do more compute than we think.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
17 / belief
I think there will be a convergence of strategies where at some point, as AI becomes more powerful, it’s going to become more or less clearer to everyone what the strategy should be.
“I think there will be a convergence of strategies where at some point, as AI becomes more powerful, it’s going to become more or less clearer to everyone what the strategy should be.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
18 / uncertainty
I do think though that one of the things not addressed here, not discussed, is that with each increase in the AI’s capabilities, I think there will be some kind of changes, but I don’t know exactly which ones, in how things are being done.
“I do think though that one of the things not addressed here, not discussed, is that with each increase in the AI’s capabilities, I think there will be some kind of changes, but I don’t know exactly which ones, in how things are being done.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
19 / uncertainty
To spell out what I mean—I don’t know whether it’s more accurate to call it a value function or reward function—but the brainstem has a directive where it’s saying, “Mate with somebody who’s more successful.
“To spell out what I mean—I don’t know whether it’s more accurate to call it a value function or reward function—but the brainstem has a directive where it’s saying, “Mate with somebody who’s more successful.”
- Speaker
- Dwarkesh Patel
- Publisher
- Dwarkesh Podcast
20 / belief
I think your point, about how from the average person’s point of view nothing is that different, will continue being true even into the singularity.
“I think your point, about how from the average person’s point of view nothing is that different, will continue being true even into the singularity.”
- Speaker
- Dwarkesh Patel
- Publisher
- Dwarkesh Podcast
21 / observation
The counterpoint is, “Hey, it is useful for the world to see powerful AI. It is useful for the world to see powerful AI because that’s the only way you can communicate it.
“The counterpoint is, “Hey, it is useful for the world to see powerful AI. It is useful for the world to see powerful AI because that’s the only way you can communicate it.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
22 / evaluation
Another thing that makes this discussion difficult is that we are talking about systems that don’t exist, that we don’t know how to build.
“Another thing that makes this discussion difficult is that we are talking about systems that don’t exist, that we don’t know how to build.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
23 / evaluation
One way to understand what you just said about not having to choose the data in pre-training is to say it’s actually not dissimilar to the 10,000 hours of practice. It’s just that you get that 10,000 hours of practice for free because it’s already somewhere in the pre-training distribution.
“One way to understand what you just said about not having to choose the data in pre-training is to say it’s actually not dissimilar to the 10,000 hours of practice. It’s just that you get that 10,000 hours of practice for free because it’s already somewhere in the pre-training distribution.”
- Speaker
- Dwarkesh Patel
- Publisher
- Dwarkesh Podcast
24 / evaluation
Pre-training is very difficult to reason about because it’s so hard to understand the manner in which the model relies on pre-training data.
“Pre-training is very difficult to reason about because it’s so hard to understand the manner in which the model relies on pre-training data.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
25 / preference
I like this idea that the real reward hacking is the human researchers who are too focused on the evals.
“I like this idea that the real reward hacking is the human researchers who are too focused on the evals.”
- Speaker
- Dwarkesh Patel
- Publisher
- Dwarkesh Podcast
26 / prediction
Now, from 2020 to 2025, it was the age of scaling—maybe plus or minus, let’s add error bars to those years—because people say, “This is amazing.
“Now, from 2020 to 2025, it was the age of scaling—maybe plus or minus, let’s add error bars to those years—because people say, “This is amazing.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
27 / prediction
I think there’ll definitely be diminishing returns because you want people who think differently rather than the same.
“I think there’ll definitely be diminishing returns because you want people who think differently rather than the same.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
28 / evaluation
The reason that the term AGI exists is, in my opinion, not so much because it’s a very important, essential descriptor of some end state of intelligence, but because it is a reaction to a different term that existed, and the term is narrow AI.
“The reason that the term AGI exists is, in my opinion, not so much because it’s a very important, essential descriptor of some end state of intelligence, but because it is a reaction to a different term that existed, and the term is narrow AI.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
29 / prediction
I think when that happens, we will see a big change in the way all AI companies approach safety.
“I think when that happens, we will see a big change in the way all AI companies approach safety.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
30 / prediction
Because what will happen as a result is that now the AI understands something, and we understand it too, because now the understanding is transmitted wholesale.
“Because what will happen as a result is that now the AI understands something, and we understand it too, because now the understanding is transmitted wholesale.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast
31 / evaluation
In terms of utility though, I think there is a thing where there is this complexity-robustness tradeoff, where complex things can be very useful, but simple things are very useful in a very broad range of situations.
“In terms of utility though, I think there is a thing where there is this complexity-robustness tradeoff, where complex things can be very useful, but simple things are very useful in a very broad range of situations.”
- Speaker
- Ilya Sutskever
- Publisher
- Dwarkesh Podcast