Evidence receipt / commitment
Published · transcript-backedSpeaker unverified: commitment
12 Jul 2024 Latent Space Benchmarks 201: Why Leaderboards > Arenas >> LLM-as-Judge
“At the same time, I won't complain too much about this because we also tend to not evaluate multi-turn for automatic benchmark.”
— Speaker unverified
Source trail
Everything needed to verify it.
- Speaker
- Speaker unverified
- Attribution
- Not verified from this transcript
- Claim type
- commitment
- Recorded
- 12 Jul 2024
- Publisher
- Latent Space
Transcript context
…So multi-turn is not tested at all. At the same time, I won't complain too much about this because we also tend to not evaluate multi-turn for automatic benchmark. So I cannot really say anything about this. The AI girlfriend community has got you there. They're very good at the multi-turn and you just need to go to OpenRouter to see which the top trending bots are. For those who don't know, a lot of this is covered in your blog posts, which I think you wrote after ICLR, which is, let's talk about LLM evaluation. You cover sort of a top-down, what you think about evals, and you even point to RavenWolf for the vibe check, who apparently blogs a lot on HuggingFace, because HuggingFace is now a blogging platform and does really good vibe checks, apparently.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.