Evidence receipt / belief
Published · transcript-backedDylan Patel: belief
2 Oct 2024 Dwarkesh Podcast @Asianometry & Dylan Patel — How the semiconductor industry actually works
“I think more reasonably, it will be like the total summation of the flops that you deliver to the model across pre-training, post-training, synthetic data for that pre-training and post-training data, as well as some of the inference time compute efficiencies.”
Source trail
Everything needed to verify it.
- Speaker
- Dylan Patel
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 2 Oct 2024
- Publisher
- Dwarkesh Podcast
Transcript context
…Right. Also, because you're doing adversarial synthetic data where the thing you're weakest at, you can make synthetic data for that, it might be way more sample efficient. So even though… Pre-training flops will be irrelevant. I actually don't think pre-training flops will be 1e30. I think more reasonably, it will be like the total summation of the flops that you deliver to the model across pre-training, post-training, synthetic data for that pre-training and post-training data, as well as some of the inference time compute efficiencies. It's more like 1e30 in total. Suppose you really do get to the world where it's worth investing…. Actually, if you're doing 1e30, is that like a trillion dollar cluster, hundred billion dollar cluster?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.