Raza Habib: evaluation

Transcript context

…so did GPT 1 and 2, or? GPD 1 and 2 had happened, but we were thinking about BERT, we were thinking about ULMfit as the first milestones that showed that this was possible. And that the kind of, it was very clear that as a result, there was going to be a huge wave of. You know, useful applications that enterprises could build on NLP that weren't previously possible. Mm hmm. But that there was still a huge lack of technical expertise and annotated data was still a big bottleneck. We were always trying to make it a lot easier for developers and for companies to adopt NLP and build useful AI products. But at the time that we started, The bottleneck was mostly, okay, do you have the right ML expertise and can you get enough annotated data? And so those were the problems we were initially helping people solve. And when GPT 3 came out I wrote a blog post about this at the time. It was very clear that this was going to be the future, that actually, because in context learning was starting to work, the amount of annotated data you would need was going to go down a lot. But until the InstructGPT papers, it still didn't feel practical. But after InstructGPT came out once you've kind of mentally done that shift, it's very hard to keep working on anything else. And so a little over a year ago we pivoted. And that was scary because we had a, we had a thing that was working. We had paying customers, like it was growing reasonably, we'd raised money. And I went to, we went to our investors at the time. And I remember having a conversation. We did a market size estimate. I actually filled out the YC application, because I think the YC application is like the simplest business model you could possibly build, right? What are you going to build? Who's it for? How are you going to make money? How big is the market? And I did the market size question, and at the time, we did it, and I was like, I reckon there are maybe three or maybe four hundred companies in the world who might need a product like this. And the assumption was that, like, okay, it's tiny today. It's mostly a small number of startups, but it will be huge in the future and that turned out clearly to be right. I didn't realize how quickly it would happen. Yeah, It's obviously surprised I think a lot of us but you were paying attention to the research when I guess a lot of people were not necessarily looking at that. Like, to my understanding, you didn't have previous NLP. knowledge, or back on, right? You, you, you, you did speech…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.