Speaker unverified: belief

Transcript context

…esources or the ability… Even the ability to get a large group together and coding is this amazing thing where it’s you and the computer and that alone, you can build really cool stuff really quickly. So for people who don’t know, Cursor is this super cool new editor that’s a fork of VS Code. It would be interesting to get your explanation of your own journey of editors. I think all of you were big fans of VS Code with Copilot. How did you arrive to VS Code and how did that lead to your journey with Cursor? Yeah, so I think a lot of us… Well, all of us were originally [inaudible 00:03:39] users. Pure Vim. Pure Vim. Yeah. No Neovim, just Pure Vim and a terminal. And at least for myself, it was around the time that Copilot came out, so 2021 that I really wanted to try it. So I went into VS Code, the only code editor in which it was available, and even though I really enjoyed using Vim, just the experience of Copilot with VS Code was more than good enough to convince me to switch. And so that kind of was the default until we started working on Cursor. And maybe we should explain what Copilot does. It’s a really nice auto complete. As you start writing a thing, it suggests one or two or three lines how to complete the thing. And there’s a fun experience in that. You know like when you have a close friendship and your friend completes your sentences? When it’s done well, there’s an intimate feeling. There’s probably a better word than intimate, but there’s a cool feeling of holy shit, it gets me. And then there’s an unpleasant feeling when it doesn’t get you. And so there’s that kind of friction. But I would say for a lot of people, the feeling that it gets me overpowers that it doesn’t. And I think actually one of the underrated aspects of Github Copilot is that even when it’s wrong, it’s a little bit annoying, but it’s not that bad because you just type another character and then maybe then it gets you, or you type another character and then it gets you. So even when it’s wrong, it’s not that bad. You can sort of iterate and fix it. I mean, the other underrated part of Copilot for me was just the first real AI product. So the first language model consumer product. So Copilot was kind of like the first killer app for LMs. Yeah. And the beta was out in 2021. Right. Okay. So what’s the origin story of Cursor? So around 2020, the scaling loss papers came out from OpenAI and that was a moment where this looked like clear predictable progress for the field where even if we didn’t have any more ideas, it looked like you could make these models a lot better if you had more compute and more data. By the way, we’ll probably talk for three to four hours on the topic of scaling loss. But just to summarize, it’s a paper in a set of papers in a set of ideas that say bigger might be better for model size and data size in the realm of machine learning. It’s bigger and better, but predictably better. Okay, that’s another topic of conversation. Yes. Yeah. as that say bigger might be better for model size and data size in the realm of machine learning. It’s bigger and better, but predictably better. Okay, that’s another topic of conversation. Yes. Yeah. So around that time for some of us, there were a lot of conceptual conversations about what’s this going to look like? What’s the story going to be for all these different knowledge worker fields about how they’re going to be made better by this technology getting better? And then I think there were a couple of moments where the theoretical gains predicted in that paper started to feel really concrete and it started to feel like a moment where you could actually go and not do a PhD if you wanted to do useful work in AI. It actually felt like now there was this whole set of systems one could build that were really useful. And I think that the first moment we already talked about a little bit, which was playing with the early beta of Copilot, that was awesome and magical. I think that the next big moment where everything kind of clicked together was actually getting early access to GPT-IV. So it was sort of end of 2022 was when we were tinkering with that model and the step-upping capabilities felt enormous. And previous to that, we had been working on a couple of different projects. Because of Copilot, because of scaling odds, because of our prior interest in the technology, we had been tinkering around with tools for programmers, but things that are very specific. So we were building tools for financial professionals who have to work within a Jupyter Notebook or playing around with can you do static analysis with these models? And then the step-up in GPT- IV felt like, look, that really made concrete the theoretical gains that we had predicted before. It felt like you could build a lot more just immediately at that point in time. And also if we were being consistent, it really felt like this wasn’t just going to be a point solution thing. This was going to be all of programming was going to flow through these models and it felt like that demanded a different type of programming environment, a different type of programming. And so we set off to build that sort of larger vision around then. There’s one that I distinctly remember. So my roommate is an IMO Gold winner and there’s a competition in the US called the PUTNAM, which is sort of the IMO for college people and it’s this math competition. It’s exceptionally good. So Shengtong and Aman I remember, sort of June of 2022, had this bet on whether the 2024 June or July you were going to win a gold medal in the IMO with models. IMO is the International Math Olympiad. Yeah, IMO is International Math Olympiad. And so Arvid and I are both also competing in it. So it was sort of personal and I remember thinking, Matt, this is not going to happen. Even though I sort of believed in progress, I thought IMO Gold, Aman is delusional. And to be honest, I mean, I was, to be clear, very wrong. But that was maybe the most prescient bet in the group. So the new results from DeepMind, it turned out that you were correct. Technically not. Technically incorrect but one point away. r, very wrong. But that was maybe the most prescient bet in the group. So the new results from DeepMind, it turned out that you were correct. Technically not. Technically incorrect but one point away. Aman was very enthusiastic about this stuff back then and before, Aman had this scaling loss T-shirt that he would wear around where it had the charts and the formulas on it. So you felt the AGI or you felt the scaling loss. Yeah, I distinctly remember there was this one conversation I had with Michael before I hadn’t thought super deeply and critically about scaling laws and he kind of posed the question, why isn’t scaling all you need or why isn’t scaling going to result in massive gains in progress? And I think I went through the stages of grief. There is anger, denial, and then finally at the end just thinking about it, acceptance. And I think I’ve been quite hopeful and optimistic about progress since. I think one thing I’ll caveat is I think it also depends on which domains you’re going to see progress. Math is a great domain especially formal theorem proving because you get this fantastic signal of actually verifying if the thing was correct. And so this means something like RL can work really, really well and I think you could have systems that are perhaps very superhuman in math and still not technically have AGI. Okay, so can we take it all the way to Cursor. And what is Cursor? It’s a fork of VS Code and VS Code is one of the most popular editors for a long time. Everybody fell in love with it. Everybody left Vim, I left DMAX for it. Sorry. So unified in some fundamental way the developer community. And then you look at the space of things, you look at the scaling laws, AI is becoming amazing and you decided okay, it’s not enough to just write an extension via VS Code because there’s a lot of limitations to that. If AI is going to keep getting better and better and better, we need to really rethink how the AI is going to be part of the editing process. And so you decided to fork VS Code and start to build a lot of the amazing features we’ll be able to talk about. But what was that decision like? Because there’s a lot of extensions, including Copilot, of VS Code that are doing sort of AI type stuff. What was the decision like to just fork VS Code? So the decision to do an editor seemed kind of self-evident to us for at least what we wanted to do and achieve because when we started working on the editor, the idea was these models are going to get much better, their capabilities are going to improve and it’s going to entirely change how you build software, both in a you will have big productivity gains but also radical and now the active building software is going to change a lot. And so you’re very limited in the control you have over a code editor if you’re a plugin to an existing coding environment and we didn’t want to get locked in by those limitations. We wanted to be able to just build the most useful stuff. Okay. Well then the natural question is, VS Code is kind of with Copilot a competitor, so how do you win? Is it basically just the speed and the quality of the features?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.