Evidence receipt / preference
Published · transcript-backedReiner Pope: preference
29 Apr 2026 Dwarkesh Podcast Reiner Pope – The math behind how LLMs are trained and served
“What that means is, given that I know I’m going to be holding something for five minutes, I would like to pick a memory that I can read every five minutes.”
Source trail
Everything needed to verify it.
- Speaker
- Reiner Pope
- Attribution
- Verified speaker
- Claim type
- preference
- Recorded
- 29 Apr 2026
- Publisher
- Dwarkesh Podcast
Transcript context
…Five minutes versus one hour. Exactly. I think this will probably end up being the drain time of the memory tier that you’re in. What that means is, given that I know I’m going to be holding something for five minutes, I would like to pick a memory that I can read every five minutes. I can read the whole memory once per five minutes, ballpark. That is the drain time of the memory. So if I take the storage capacity over storage bandwidth, I would like this to be equal to five minutes. We did this calculation for HBM. For HBM, we know that this number is 20 milliseconds. So HBM is much too small. DDR could be about an order of magnitude or two off from this, so this is probably on the order of seconds, like 1 to 10 seconds. I don’t have these numbers memorized, but generally, as you go to slower tiers, flash is plausibly on the order of one minute. And then spinning disk, which is massively different, is on the order of one hour. So this might actually identify the tiers of flash and spinning disk. Sorry, why is this the calculation? This is the storage capacity divided by the bandwidth?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.