High Signal Podcasts Evidence ledger
Method
Browse
← Back to evidence

Evidence receipt / belief

Published · transcript-backed

Ankur Goyal: belief

11 Oct 2024 Latent Space Production AI Engineering starts with Evals — with Ankur Goyal of Braintrust

“I think, to me, it seems very unlikely that you and me sort of like sipping an espresso and thinking about how different personified roles of people should interact with each other and stuff.”

— Ankur Goyal

Source trail

Everything needed to verify it.

Speaker
Ankur Goyal
Attribution
Verified speaker
Claim type
belief
Recorded
11 Oct 2024
Publisher
Latent Space

Transcript context

…There were two breakers, Dylan and us. George was the first loud enough person to make noise about it. Prior to that, a lot of people were building these round-robin routers that were like, you know, and you look at that and you're like, okay, I'm pretty sure if you train a model to do this problem and you vertically integrate that into the LLM itself, it's going to be better. And that happened with GPT-4. And I think O1 is going to do that to agentic frameworks as well. I think, to me, it seems very unlikely that you and me sort of like sipping an espresso and thinking about how different personified roles of people should interact with each other and stuff. It seems like that is just going to get pushed into the model. That was the main takeaway for me. I think that you are very perceptive in your mental modeling of me, because I do disagree 15-25%. Obviously, they can do things that we cannot, but you as a business always want more control than OpenAI will ever give you. They're charging you for thousands of reasoning tokens and you can't see it. That's ridiculous. Come on.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.

Search evidence