High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works
← All source episodes

Lex Fridman Podcast / episode intelligence

#447 – Cursor Team: Future of Programming with AI

6 Oct 2024 20 published claims 0 attributable people
Open original episode ↗ Read the evidence

Speakers in the public record

Claim mix

belief 13evaluation 3prediction 2recommendation 1commitment 1

Evidence policy

Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.

Claim ledger

The useful parts, with receipts.

20 published records

01 / evaluation

I think that there are also cool academic ideas, stuff we’ve tried out internally, but also the field is grappling with writ large about, can you get language models to a place where you can actually just have the model itself understand a new corpus of information?

“I think that there are also cool academic ideas, stuff we’ve tried out internally, but also the field is grappling with writ large about, can you get language models to a place where you can actually just have the model itself understand a new corpus of information?”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

02 / prediction

Meaning, it’s going to be hard to continue scaling up this regime. So scaling up test time compute is an interesting way, if now increasing the number of inference time flops that we use but still getting… Yeah, as you increase the number of flops you use inference time getting corresponding improvements in the performance of these models.

“Meaning, it’s going to be hard to continue scaling up this regime. So scaling up test time compute is an interesting way, if now increasing the number of inference time flops that we use but still getting… Yeah, as you increase the number of flops you use inference time getting corresponding improvements in the performance of these models.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

03 / recommendation

I think for the more aggressive things where you’re making larger changes that take longer periods of time, you’ll probably want to do this in some sandbox remote environment and that’s another incredibly tricky problem of how do you exactly reproduce or mostly reproduce to the point of it being effectively equivalent for running code the user’s environment with this remote sandbox.

“I think for the more aggressive things where you’re making larger changes that take longer periods of time, you’ll probably want to do this in some sandbox remote environment and that’s another incredibly tricky problem of how do you exactly reproduce or mostly reproduce to the point of it being effectively equivalent for running code the user’s environment with this remote sandbox.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

04 / evaluation

There’s this interesting thing where if you look at language model loss on different domains, I believe the bits per byte, which is a kind of character normalize loss for code is lower than language, which means in general there are a lot of tokens in code that are super predictable, a lot of characters that are super predictable.

“There’s this interesting thing where if you look at language model loss on different domains, I believe the bits per byte, which is a kind of character normalize loss for code is lower than language, which means in general there are a lot of tokens in code that are super predictable, a lot of characters that are super predictable.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

06 / evaluation

I think there’s no model that Pareto dominates others, meaning it is better in all categories that we think matter, the categories being speed, ability to edit code, ability to process lots of code, long context, a couple of other things and coding capabilities.

“I think there’s no model that Pareto dominates others, meaning it is better in all categories that we think matter, the categories being speed, ability to edit code, ability to process lots of code, long context, a couple of other things and coding capabilities.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

07 / belief

Yeah, I think you see shallow copies of apply elsewhere and it just breaks most of the time because you think you can try to do some deterministic matching and then it fails at least 40% of the time and that just results in a terrible product experience.

“Yeah, I think you see shallow copies of apply elsewhere and it just breaks most of the time because you think you can try to do some deterministic matching and then it fails at least 40% of the time and that just results in a terrible product experience.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

08 / belief

The last category I think is, I guess the main one that it feels like the big labs are doing for synthetic data, which is producing text with language models that can then be verified easily.

“The last category I think is, I guess the main one that it feels like the big labs are doing for synthetic data, which is producing text with language models that can then be verified easily.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

10 / belief

Why do you think it’s different than cloud providers? Because I think a lot of this data would never have gone to the cloud providers in the first place where this is often… You want to give more data to the AI models, you want to give personal data that you would never have put online in the first place to these companies or to these models.

“Why do you think it’s different than cloud providers? Because I think a lot of this data would never have gone to the cloud providers in the first place where this is often… You want to give more data to the AI models, you want to give personal data that you would never have put online in the first place to these companies or to these models.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

11 / belief

Then after we had a good model, I think there’ve been a lot of effort to make the inference fast for having a good experience, and we’ve been starting to incorporate… I mean, Michael sort of mentioned this ability to jump to different places and that jump to different places I think came from a feeling of once you accept an edit, it’s like man, it should be just really obvious where to go next.

“Then after we had a good model, I think there’ve been a lot of effort to make the inference fast for having a good experience, and we’ve been starting to incorporate… I mean, Michael sort of mentioned this ability to jump to different places and that jump to different places I think came from a feeling of once you accept an edit, it’s like man, it should be just really obvious where to go next.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

12 / belief

The interesting work that I think has been done is figuring out how to properly train the process… Or the interesting work that has been open sourced and people I think talk about is how to train the process reward models, maybe in a more automated way.

“The interesting work that I think has been done is figuring out how to properly train the process… Or the interesting work that has been open sourced and people I think talk about is how to train the process reward models, maybe in a more automated way.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

13 / belief

Actually, I would say you swoop in and you get all the information, all the little heuristics, all the little parameters, all the parameters that define how the thing is trained.

“Actually, I would say you swoop in and you get all the information, all the little heuristics, all the little parameters, all the parameters that define how the thing is trained.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

15 / belief

You can wax poetic about moats this and brand that and this is our advantage, but I think in the end, just if you stop innovating on the product, you will lose. That’s also great for startups, that’s great for people trying to enter this market because it means you have an opportunity to win against people who have lots of users already by just building something better.

“You can wax poetic about moats this and brand that and this is our advantage, but I think in the end, just if you stop innovating on the product, you will lose. That’s also great for startups, that’s great for people trying to enter this market because it means you have an opportunity to win against people who have lots of users already by just building something better.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

16 / belief

I don’t think we think that that’s the case because a lot of programming, a lot of the value is in iterating, or you don’t actually want to specify something upfront because you don’t really know what you want until you have seen an initial version and then you want to iterate on that and then you provide more information.

“I don’t think we think that that’s the case because a lot of programming, a lot of the value is in iterating, or you don’t actually want to specify something upfront because you don’t really know what you want until you have seen an initial version and then you want to iterate on that and then you provide more information.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

17 / belief

I think yeah, because even with all this compute and all the data you could collect in the world, I think you really are ultimately limited by not even ideas, but just really good engineering.

“I think yeah, because even with all this compute and all the data you could collect in the world, I think you really are ultimately limited by not even ideas, but just really good engineering.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

18 / belief

Fundamentally, I think one of the things that draws a lot of people to building stuff on computers is this insane iteration speed, where in other disciplines you might be sort of gate capped by resources or the ability… Even the ability to get a large group together and coding is this amazing thing where it’s you and the computer and that alone, you can build really cool stuff really quickly.

“Fundamentally, I think one of the things that draws a lot of people to building stuff on computers is this insane iteration speed, where in other disciplines you might be sort of gate capped by resources or the ability… Even the ability to get a large group together and coding is this amazing thing where it’s you and the computer and that alone, you can build really cool stuff really quickly.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

19 / belief

What’s the story going to be for all these different knowledge worker fields about how they’re going to be made better by this technology getting better? And then I think there were a couple of moments where the theoretical gains predicted in that paper started to feel really concrete and it started to feel like a moment where you could actually go and not do a PhD if you wanted to do useful work in AI.

“What’s the story going to be for all these different knowledge worker fields about how they’re going to be made better by this technology getting better? And then I think there were a couple of moments where the theoretical gains predicted in that paper started to feel really concrete and it started to feel like a moment where you could actually go and not do a PhD if you wanted to do useful work in AI.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast

20 / belief

You thought through everything, which you didn’t actually think through everything. But I think for that particular system, we’ve… So for concrete details, the thing we do is obviously we upload when… We chunk up all of your code, and then we send up the code for embedding and we embed the code.

“You thought through everything, which you didn’t actually think through everything. But I think for that particular system, we’ve… So for concrete details, the thing we do is obviously we upload when… We chunk up all of your code, and then we send up the code for embedding and we embed the code.”
Speaker
Not verified from transcript
Publisher
Lex Fridman Podcast
Search evidence