Lenny Rachitsky: evaluation

Transcript context

…And then on the engineering side, it's very variable because like I said before, it matters how big and how complex the code base is. And so if you're building a totally new Greenfields code base or you're building an app for a new platform, then we're seeing those pretty aggressive gains, but in very complex code bases that already exist, those gains are not quite there yet. That's amazing. And whenever I hear a stat like this, I think an important element that people need to think about is this is the worst it will ever be. This is the lowest, this is now the baseline. And so it may not sound that crazy yet, but it's going to get crazy. Okay, the other thing that you talked about is Goose, you haven't explained what Goose is. This is a huge deal. Explain what Goose is and how important this has become to you guys. So Goose is a general purpose AI agent. So you can think of it as a desktop tool or a program that you can download and install on your computer and then it has a UI. You can talk to it just like a chatbot and you can say anything from, "Hey Goose, organize my photos by category, and it has the ability to look within your photos and if there are a lot of trees, it'll organize them as nature photos. And there are a lot of people, it'll organize them as portraiture." All of this sort of stuff to writing software for you. So it can do all of these tasks, and the way we've been able to do this is through something called a model context protocol or the MCP, which a lot of your listeners might've heard. And this is something that Anthropic came up with that we were a very early contributor to. And the model context protocol is very simply just a set of formalized wrappers around existing tools or existing capabilities. So if you have tools that you use in the enterprise, be it Salesforce or be it Snowflake or SQL, any of these things, you can wrap them in the MCP and then it exposes them to your LLM to be able to manipulate. So until that point, the LLMs were not really able to do much other than chat, but Goose gives these brains arms and legs to go out and act in our digital world, and that's where we find it's had most impact and it's built on this fairly open protocol that anyone can implement. There have been an explosion of MCPs. Goose is entirely open source, by the way, so any of you can download it and extend it, write your own MCPs, and that's been our core successes through Goose.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.