Sherwin Wu: recommendation

Transcript context

…I don't know if it's that hot of a take. I think the main thing here is, so obviously you should talk to your customers. It's like you still talk to customers. I just think the AI field, especially what I've seen over the last three years working on the API and seeing all that evolve, is the field and the models themselves are just changing so, so quickly, they tend to disrupt themselves, especially around the tooling and the scaffolding space. So there's this quote that I read actually earlier this week, it's from an X article by this guy named Nicolas, who's the founder of a startup called Fintool, where I think he was sharing a lot of the best practices that he has learned through building AI agents for financial services, I think at his startup Fintool. And this phrase that I thought was really good, which is the models will eat your scaffolding for breakfast. If you rewind back to 2022, right when ChatGPT launched, these models are pretty raw and there was like all this product scaffolding and things, especially in the developer space, to basically try and steer the model and build a scaffolding around it to get it to do what you want. Like agent frameworks, there's like vector stores I think was like really popular back then and just like a whole smattering of tools here. And as you've kind of seen the field play out, the models have just changed so much and gotten so much better that they ended up literally eating some of the scaffolding. And I think this is even true today. So I think that the article from Nicolas actually, the current scaffolding which is fashionable is Skills, files-based context management. I could see a world where at some point that's no longer useful, where the model can actually manage all that themselves, or there might be, it's hard to predict, but might move on to some new paradigm where you'll already need this file-based Skills type thing. You have literally seen this play out where like the agent frameworks I think are a little less useful now. There was a period of time in like 2023 where we thought vector stores is going to be like the main way for you to bring organizational context into the models and you need to vectorize and embed every bit of your corpuses and then you need to do all this work to figure out the vector search to optimize that to fill out the right information at the right time. All of that is scaffolding because the model was not good enough. And turns out, in this case, it turns out as the models get better, a better approach is actually to take out a lot of that logic and trust the model and give it a set of tools for search. It doesn't need to be a vector store. You could actually just hook it up to any type of search. It could literally be files on a file system like Skills and AGENTS.md to kind of steer it as well. Obviously there's still a place for vector stores. d actually just hook it up to any type of search. It could literally be files on a file system like Skills and AGENTS.md to kind of steer it as well. Obviously there's still a place for vector stores. I know a lot of companies still using it, but the entire scaffolding around that and building an entire ecosystem around that and assuming that's the only scaffolding that you need has really changed. And so tying this back to the like, you don't always have to listen to your customers. Because the field is changing so much at any point in time, a lot of people are kind of in this local maximum. And if you just blindly listen to your customers, they'll be like, "Yeah, I want a better vector store. I want a better agent framework for this." And if you had just kind of only chased down that path, it actually would've led you to build something that again is the local maxima. Whereas as the models get better, we've had to reinvent and kind of rethink the right abstractions and the right tools and frameworks to build around these models. And the cool/exciting/kind of crazy annoying part is it's a moving target. And so yeah, the current smattering of tools and frameworks right now will likely need to evolve and change pretty significantly over time as the models get smarter and better. But that is just the nature of building in this space. I think that's what makes it exciting, but it also means when you talk to customers, you kind of need to balance the exact feedback that they want with where you think the models are going and where you think things will trend over the next one to two years. It's interesting how this is, the bitter lesson is this big lesson that AI and ML folks learned, which is just like, the less you overcomplicate, the less logic you add to machine learning and to AI, the more it'll be able to scale and grow and just take it all away and let it just compute basically. Just give it more power to get smarter on its own.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.