Evidence receipt / belief
Published · transcript-backedShawn Wang: belief
11 Oct 2024 Latent Space Production AI Engineering starts with Evals — with Ankur Goyal of Braintrust
“I think that you are very perceptive in your mental modeling of me, because I do disagree 15-25%.”
Source trail
Everything needed to verify it.
- Speaker
- Shawn Wang
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 11 Oct 2024
- Publisher
- Latent Space
Transcript context
…a lot of people were building these round-robin routers that were like, you know, and you look at that and you're like, okay, I'm pretty sure if you train a model to do this problem and you vertically integrate that into the LLM itself, it's going to be better. And that happened with GPT-4. And I think O1 is going to do that to agentic frameworks as well. I think, to me, it seems very unlikely that you and me sort of like sipping an espresso and thinking about how different personified roles of people should interact with each other and stuff. It seems like that is just going to get pushed into the model. That was the main takeaway for me. I think that you are very perceptive in your mental modeling of me, because I do disagree 15-25%. Obviously, they can do things that we cannot, but you as a business always want more control than OpenAI will ever give you. They're charging you for thousands of reasoning tokens and you can't see it. That's ridiculous. Come on. Well, it's ridiculous until it's not, right? I mean, it was ridiculous to GPT-3 too.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.