Evidence receipt / belief
Published · transcript-backedJosh Albrecht: belief
25 Jun 2024 Latent Space State of the Art: Training >70B LLMs on 10,000 H100 clusters
“I mean, one of them, which I think is probably pretty common, like common knowledge by this point.”
Source trail
Everything needed to verify it.
- Speaker
- Josh Albrecht
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 25 Jun 2024
- Publisher
- Latent Space
Transcript context
…Yeah, excellent. And then, you know, you so you have all the hardware set up and now you're firing it up for a single node. There's a long description that you guys have about just like monitoring the MFU, right? And what each situation might look might be indicative of. One of the most interesting things to me that I saw from here is like, you know, if training immediately starts off at 60 to 80% MFU, something's wrong. But like, you know, like what what are like, you know, some anecdotes or, you know, notable scenarios here that you might you might call out as maybe counterintuitive or super interesting. There's just so many of them. I mean, one of them, which I think is probably pretty common, like common knowledge by this point. But like we did have a sort of like which one was this exactly?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.