Dwarkesh Patel: evaluation

Transcript context

…So it actually happened in a pretty simple way, which is that we had our own coding models, which were good at coding. Around the beginning of 2025, I said, “I think the time has come where you can have nontrivial acceleration of your own research if you’re an AI company by using these models.” Of course, you need an interface, you need a harness to use them. So I encouraged people internally. I didn’t say this is one thing that you have to use. I just said people should experiment with this. I think it might have been originally called Claude CLI, and then the name eventually got changed to Claude Code. Internally, it was the thing that everyone was using and it was seeing fast internal adoption. I looked at it and I said, “Probably we should launch this externally, right?” It’s seen such fast adoption within Anthropic. Coding is a lot of what we do. We have an audience of many, many hundreds of people that’s in some ways at least representative of the external audience. So it looks like we already have product market fit. Let’s launch this thing. And then we launched it. I think just the fact that we ourselves are kind of developing the model and we ourselves know what we most need to use the model, I think it’s kind of creating this feedback loop. I see. In the sense that you, let’s say a developer at Anthropic is like, “Ah, it would be better if it was better at this X thing. ” Then you bake that into the next model that you build. That’s one version of it, but then there’s just the ordinary product iteration. We have a bunch of coders within Anthropic, they use Claude Code every day and so we get fast feedback. That was more important in the early days. Now, of course, there are millions of people using it, and so we get a bunch of external feedback as well. But it’s just great to be able to get kind of fast internal feedback. I think this is the reason why we launched a coding model and didn’t launch a pharmaceutical company. My background’s in biology, but we don’t have any of the resources that are needed to launch a pharmaceutical company.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.