01 / recommendation
Built CS231n.
“Earlier on, I built CS231n at Stanford, which I think was the first deep learning class at Stanford, which became very popular.”
- Speaker
- Andrej Karpathy
- Publisher
- Dwarkesh Podcast
Dwarkesh Podcast / episode intelligence
Speakers in the public record
Claim mix
Evidence policy
Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.
Claim ledger
47 published records
01 / recommendation
“Earlier on, I built CS231n at Stanford, which I think was the first deep learning class at Stanford, which became very popular.”
02 / belief
“I think that’s right. If you’re sticking to the realm of bits, bits are a million times easier than anything that touches the physical world.”
03 / uncertainty
“The easiest way I can describe it is we’re trying to build the Starfleet Academy. I don’t know if you’ve watched Star Trek.”
04 / belief
“The first one I would say is culture and LLMs having a growing repertoire of knowledge for their own purposes.”
05 / belief
“I am betting a bit implicitly on some of the timelessness of human nature. It will be desirable to do all these things, and I think people will look up to it as they have for millennia.”
06 / prediction
“I expect a similar thing from AI where it’s not like there’s going to be a single moment where we’ve made the crucial invention.”
07 / belief
“I still think you’re presupposing some discrete jump that has no historical precedent that I can’t find in any of the statistics and that I think probably won’t happen.”
08 / belief
“You just take all the course materials and then I think you could serve a very good automated TA for the student when they have more basic questions or something like that.”
09 / belief
“I think you’re basically inventing college from first principles for the tools that are available today and just selecting for people who have the motivation and the interest of really engaging with material.”
10 / belief
“Text is maybe a lot more flowery, and there’s a lot more entropy in text, I would say.”
11 / belief
“Just to throw the opposite argument against you, my expectation is that it blows up because I think true AGI—and I’m not talking about LLM coding bots, I’m talking about actual replacement of a human in a server—is qualitatively different from these other productivity-improving technologies because it’s labor itself.”
12 / observation
“The way to synchronize gradients between them is to use a Distributed Data Parallel container of PyTorch, which automatically as you’re doing the backward, it will start communicating and synchronizing gradients.”
13 / belief
“There’s a lot of really smart people who are ready to make use of the resources and do this period of catch-up because we’ve had this discontinuity, and I think AI might be similar.”
14 / uncertainty
“In terms of learning, I don’t know that I necessarily found something that I learned from it.”
15 / commitment
“If I have 20 minutes, I will copy-paste my entire repo and I go to GPT-5 Pro, the oracle, for some questions.”
16 / uncertainty
“Then adults are somewhere in between, where they don’t have the flexibility of childhood learning, but they can memorize facts and information in a way that is harder for kids. I don’t know if there’s something interesting about that spectrum.”
17 / evaluation
“I guess they just don’t work as well empirically because right now the models are collapsed.”
18 / belief
“One example that’s prominently in my mind, this was probably public, if you’re using an LLM judge for a reward, you just give it a solution from a student and ask it if the student did well or not.”
19 / prediction
“We expect more and more autonomous entities over time that are doing a lot of the digital work and then eventually even the physical work some amount of time later.”
20 / prediction
“If somebody asks how long continual learning will take, I have no prior about whether this is a project that should take 5 years, 10 years, or 50 years.”
21 / prediction
“I’m looking for an autonomy slider. I expect that we are not going to instantly replace people.”
22 / belief
“There’s been evidence that that’s already been happening generally in companies that have been adopting AI, which I think is quite surprising.”
23 / observation
“If you buy the Sutton perspective that the crux of intelligence is animal intelligence… The quote he said is “If you got to the squirrel, you’d be most of the way to AGI.”
24 / belief
“We could also look at it with respect to how many times we think certain intelligence has individually sprung up.”
25 / belief
“In my mind, education is the very difficult technical process of building ramps to knowledge. In my mind, nanochat is a ramp to knowledge because it’s very simple.”
26 / belief
“LLMs don’t really have culture right now and it’s one of the impediments I would say.”
27 / uncertainty
“We’re really far into a territory where I don’t know what this looks like, but if I were to write sci-fi novels, they would look along the lines of not even a single entity that takes over everything, but multiple competing entities that gradually become more and more autonomous.”
28 / belief
“Then I think you get away with a much smaller model because it’s a much better dataset and you could train it on it.”
29 / belief
“I think that’s probably holding back the neural networks overall because it’s getting them to rely on the knowledge a little too much sometimes.”
30 / commitment
“I think there will be a transitional period where we are going to be able to be in the loop and advance things if we understand a lot of stuff.”
31 / uncertainty
“I mean, come on, right? I don’t know. At some point it should take at least a billion knobs to do something interesting.”
32 / belief
“Even the early iPhone didn’t have the App Store, and it didn’t have a lot of the bells and whistles that the modern iPhone has. So even though we think of 2008, when the iPhone came out, as this major seismic change, it’s actually not.”
33 / belief
“I care not just about all the Dyson spheres that we’re going to build and that AI is going to build in a fully autonomous way, I care about what happens to humans.”
34 / prediction
“For the most part, yeah. I expect the datasets to get much, much better. When you look at the average datasets, they’re extremely terrible.”
35 / belief
“I think that’s the most likely outcome, that there will be a gradual loss of understanding.”
36 / belief
“I think you hinted that it’s a very fundamental problem, it won’t be easy to solve.”
37 / belief
“The end is not near yet because when we’re talking about self-driving, usually in my mind it’s self-driving at scale.”
38 / evaluation
“The paper that blew my mind was InstructGPT, because it pointed out that you can take the pretrained model, which is autocomplete, and if you just fine-tune it on text that looks like conversations, the model will very rapidly adapt to become very conversational, and it keeps all the knowledge from pre-training.”
39 / commitment
“I have a hard hat on, and I’m just observing that we’re not going to do evolution, because I don’t know how to do that.”
40 / evaluation
“Just a lot of it. So maybe one way to think about it, I don’t know if this is the best way, but I almost feel like — again, making these analogies imperfect as they are — we’ve stumbled by with the transformer neural network, which is extremely powerful, very general.”
41 / evaluation
“” The Atari deep reinforcement learning shift in 2013 or so was part of that early effort of agents, in my mind, because it was an attempt to try to get agents that not just perceive the world, but also take actions and interact and get rewards from environments.”
42 / evaluation
“In fact, I think the paper was even stronger because they hardcoded the weights of a neural network to do gradient descent through attention and all the internals of the neural network.”
43 / evaluation
“We have some very early agents that are extremely impressive and that I use daily—Claude and Codex and so on—but I still feel there’s so much work to be done.”
44 / observation
“The first concession that people make all the time is they just take out all the physical stuff because we’re just talking about digital knowledge work.”
45 / evaluation
“I would say nanochat is not an example of those because it’s a fairly unique repository.”
46 / recommendation
“I have a whole rant on how everyone should learn physics in early school education because early school education is not about accumulating knowledge or memory for tasks later in the industry.”
47 / prediction
“Why don’t you do it today? The reason you don’t do it today is because they just don’t work.”