Evidence receipt / belief
Published · transcript-backedReiner Pope: belief
29 Apr 2026 Dwarkesh Podcast Reiner Pope – The math behind how LLMs are trained and served
“I think starting with equalizing in cost is right, but depending on how you model the cost, this comes close to equalizing in data.”
Source trail
Everything needed to verify it.
- Speaker
- Reiner Pope
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 29 Apr 2026
- Publisher
- Dwarkesh Podcast
Transcript context
…This is all quite interesting. I never thought about it in terms of equalizing data. I think starting with equalizing in cost is right, but depending on how you model the cost, this comes close to equalizing in data. So for GPT to be trained optimally, every single user who uses GPT-5, the total amount of tokens that they stream should equal the total amount that has gone into pre-training. And the total amount of tokens that have gone into pre-training is the sum of all human knowledge. Each model should generate the sum of human knowledge on the output that it gets on the input.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.