Dwarkesh Podcast / episode intelligence
Speakers in the public record
Claim mix
Evidence policy
Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.
Claim ledger
24 published records
01 / evaluation
“Generally, people will go a little bit larger than this. They don’t really want to be exactly at the balance point because real-world efficiencies aren’t as good as a roofline analysis would say.”
02 / evaluation
“It is simply that I need to read all of my total parameters from memory into the chips, and that takes a certain amount of time. If I use all of my memory bandwidth, I can’t do any better than that.”
03 / belief
“Ok, so I was asking you, have neural networks actually been used for cryptography? And we realized it may be better to just do this on the blackboard.”
04 / belief
“I don’t have that information, but I think a good ballpark is a 33% split between each of them.”
05 / belief
“If you look at Hopper, you had eight Hoppers, and I think that’s 640 gigabytes as of 2022.”
06 / belief
“I think that’s a pretty good assumption. If you look at the numbers, it might also turn out that it’s one tier down, and it’s DDR versus flash.”
07 / commitment
“I can motivate the batch at least a little bit. We will see exactly why batch is such a favorable optimization.”
08 / belief
“I think if you do it with a spreadsheet and actually model it out, you might notice when the money’s going down the drain.”
09 / belief
“I think often the number of active parameters could be in the range of a hundred billion, something like that.”
10 / belief
“I think starting with equalizing in cost is right, but depending on how you model the cost, this comes close to equalizing in data.”
11 / belief
“I think we just have to make some real-world assumptions here in order to do that.”
12 / commitment
“We will directly jump to the ratio of the time spent on scale-up over the time spent on scale-out.”
13 / belief
“Sorry, I think the way I said it was super garbled. Just for the audience, forward plus backwards per parameter is 6.”
14 / belief
“If we think that the memory in a rack is a bottleneck, then there’s a constraint on how fast we can go.”
15 / recommendation
“If you were designing your hardware, you could say, “I didn’t need that much memory because I don’t need the weights to fit in one rack.”
16 / evaluation
“I think that indicates that this is the reasonably balanced cost point, and going massively beyond that would be cost-prohibitive.”
17 / evaluation
“I can’t use different pipeline stages in parallel because they’re not running at the same time, but I can use all the GPUs in my scale-up domain in parallel to load the weights.”
18 / evaluation
“I’m kind of shocked to see spinning disk being used at all, because it’s such an old technology.”
19 / evaluation
“Because of the ability to do pipelining, a rack at least should not be a constraint on your ability to fit the model parameters.”
20 / evaluation
“If I think about what that means, it means I actually have time to read the HBM twice.”
21 / preference
“What that means is, given that I know I’m going to be holding something for five minutes, I would like to pick a memory that I can read every five minutes.”
22 / evaluation
“Oh, interesting. This is sort of obvious, but the difference between micro-batch and batch doesn’t matter at all in inference because you can just call it whatever you want.”
23 / evaluation
“If I think about what cost means… To run this inference, I’m going to use the GPU for a certain number of seconds, like one millisecond or 20 milliseconds.”
24 / evaluation
“A way to think of this is that if I have too many of these things sitting in my HBM, if I fill up my HBM with just KV caches that I’m not using, I can’t use that GPU.”