Ankur Goyal: belief

Transcript context

…screwed. And I got screwed, not necessarily a bad way, but I sort of felt that twice in a short period of time. I think that shook out of me, that temptation as an engineer that you have to say, GPT-4.0 is good at this, but models will never be good at that. So let me try to build software that works around that. I think probably you might actually disagree with this, and I wouldn't say that I have a perfectly strong structural argument about this. I'm open to debate, and I might be totally wrong, but I think one of the things that felt obvious to me and somewhat vindicated by O1 is that there's a lot of code and paths that people went down with GPT-4. 0 to achieve this idea of more complex reasoning, and I think agentic frameworks are kind of like a little Cambrian explosion of people trying to work around the fact that GPT-4.0 or related models have somewhat limited reasoning capabilities. I look at that stuff and writing graph code that returns edge indirections and all this, it's like, oh my god, this is so complicated. It feels very clear to me that this type of logic is going to be built into the model. Anytime there is control flow complexity or uncertainty complexity, I think the history of AI has been to push more and more into the model. In fact, no one knows whether this is true or whatever, but GPT-4.0 was famously a mixture of experts. You mentioned it on our podcast.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.