Speaker unverified: belief

Transcript context

…Yeah. Perhaps we can transition to evals. We've had some questions about the 400B details that we want to disclose, you know, by the time this podcast comes out, you know, we'll have disclosed them. Yeah. I think last time you disclosed like the evals while you were still training, what should people know about the high level headlines for the new Llama 3? At a high level, it's the best open source model ever. It's better than GPT-4. I mean, what version, but by far compared to the version originally released, even now, I think there's maybe the last clouds on a 3. 5 and GPT-4.0 that are performing it. And that's it. Period. For the 405B, that's a flagship, that's a pretty good model. Not yet the number one. We still have a journey to get there. For the 7TB and 7B, they are like world-class models for this size, for general models. And are the benchmark numbers from the initial checkpoint still right? So the April 15 checkpoint, MMLU on Instruct is like 86, GPUA 48, HumanEval 84, GSMAK 94, and that's 57.8. Is this still roughly the same performance or, you know, I haven't seen the numbers yet either. We're just breaking the news right now.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.