High Signal Podcasts Evidence ledger
Method
Browse
← Back to evidence

Evidence receipt / evaluation

Published · transcript-backed

Mia Glaese: evaluation

23 Feb 2026 Latent Space ⚡️The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data

“I think like complexity, however we can sort of like quantify it, is really important to understand like where our motive are, are are getting to.”

— Mia Glaese

Source trail

Everything needed to verify it.

Speaker
Mia Glaese
Attribution
Verified speaker
Claim type
evaluation
Recorded
23 Feb 2026
Publisher
Latent Space

Transcript context

…Yeah. Okay. So there, there are proxies for some amount of increasing [00:21:00] capacity that we wanna measure. I think that’s a good thing. I think the only other sort of major player in this field is meter, which has done the sort of long grass and congrats. You guys have completely destroyed the curve for that. Any takes on that? Obviously you’ve come up really well, so like it looks good, but I don’t know if like that approach is something that you wanna incorporate in your own work, making it else. This is the long autonomy pass eval. You’re, yeah, yeah. No, from we are, and, and, and we, we, we work with, with meter on these evaluations. So like we, we do appreciate them. I think then they’re using time, right? They’re not using money. So I think like that was a, your question. I think like complexity, however we can sort of like quantify it, is really important to understand like where our motive are, are are getting to. Okay. Com complexity is the abstract thing and it projects down the time, projects down to. Story points, whatever dollars. Great. One last question on just like, just the overall preparedness framework is you know, I was actually kind of looking at, people mentioned the preparedness framework a lot. I don’t think it’s well explained to a lot of people. And you actually have a nice website where it’s like I think it’s like test and like inform and teach something. And I, I feel like you, you actually do a lot of work there and, and I don’t know if you wanna talk about how the preparedness framework applies.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.

Search evidence