01 / disagreement
Progress has kind of stalled. And basically we realized that this is because the eval is effectively saturated and also highly contaminated.
“Progress has kind of stalled. And basically we realized that this is because the eval is effectively saturated and also highly contaminated.”
- Speaker
- Olivia Watkins
- Episode
- ⚡️The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data
- Publisher
- Latent Space