Evidence receipt / belief
Published · transcript-backedSpeaker unverified: belief
1 Feb 2026 Lex Fridman Podcast #490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
“I think the only fair way to evaluate an LLM is to have a new benchmark that is after the cutoff date when the model was deployed.”
— Speaker unverified
Source trail
Everything needed to verify it.
- Speaker
- Speaker unverified
- Attribution
- Not verified from this transcript
- Claim type
- belief
- Recorded
- 1 Feb 2026
- Publisher
- Lex Fridman Podcast
Transcript context
…I think that that could be like a model issue rather than a general issue. It’s not even malicious by the developers of the LLM, like, “Hey, we want to cheat at that benchmark.” It’s just it has seen something at some point. I think the only fair way to evaluate an LLM is to have a new benchmark that is after the cutoff date when the model was deployed. Can we lay out what would be the recipe of all the things that go into post-training? And you mentioned RLVR was a really exciting, effective thing. Maybe we should elaborate. RLHF still has a really important component to play. What kind of other ideas are there on post-training?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.