Evidence receipt / commitment
Published · transcript-backedMark Zuckerberg: commitment
18 Apr 2024 Dwarkesh Podcast Mark Zuckerberg — Llama 3, $10B models, Caesar Augustus, & 1 GW datacenters
“We built two, I think 22,000 or 24,000 clusters that are the single clusters that we have for training the big models, obviously across a lot of the stuff that we do.”
Source trail
Everything needed to verify it.
- Speaker
- Mark Zuckerberg
- Attribution
- Verified speaker
- Claim type
- commitment
- Recorded
- 18 Apr 2024
- Publisher
- Dwarkesh Podcast
Transcript context
…So you have all these GPUs. I think you said 350,000 by the end of the year. That's the whole fleet. We built two, I think 22,000 or 24,000 clusters that are the single clusters that we have for training the big models, obviously across a lot of the stuff that we do. A lot of our stuff goes towards training Reels models and Facebook News Feed and Instagram Feed. Inference is a huge thing for us because we serve a ton of people. Our ratio of inference compute required to training is probably much higher than most other companies that are doing this stuff just because of the sheer volume of the community that we're serving. In the material they shared with me before, it was really interesting that you trained it on more data than is compute optimal just for training. The inference is such a big deal for you guys, and also for the community, that it makes sense to just have this thing and have trillions of tokens in there.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.