High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works

Public evidence record

Alessio Fanelli

Partner · Decibel

Claims
224
Episodes
65
Shows
1
Named items
3

Books, apps, and tools

The evidenced stack.

Browse the grouped index →

tool / uses

Zo

“I think there’s been one thing, I use another thing called zo, which is kinda like a cloud computer plus agent.”

Latent Space · 17 Mar 2026

Evidence receipt · Source ↗

tool / likes

VCR

“First of all, you have very good support for mocking in unit tests, which is something that a lot of other frameworks don't do. So, you know, my favorite Ruby library is VCR because it just, you know, it just lets me store the HTTP requests and replay them.”

Latent Space · 6 Feb 2025

Evidence receipt · Source ↗

other / likes

Simulative AI

“Yeah, I'm a big fan of Simulative AI. We had a summer of Simulative AI. Another term we're trying to coin.”

Latent Space · 27 Sept 2024

Evidence receipt · Source ↗

Claim ledger

What Alessio said.

224 transcript-backed records

01 / belief

I think most people have heard of Applied Intuition tied to YC when it was first started, and then you were kinda in stealth for a long time, so maybe just give people the high-level overview of what it is today, and then we’ll dive into the different pieces.

“I think most people have heard of Applied Intuition tied to YC when it was first started, and then you were kinda in stealth for a long time, so maybe just give people the high-level overview of what it is today, and then we’ll dive into the different pieces.”
Speaker
Alessio Fanelli
Publisher
Latent Space

02 / evaluation

I was gonna say, to me, driving feels like a great next token prediction thing because you’re kinda like on a path and like, it doesn’t really matter what you’ve done before.

“I was gonna say, to me, driving feels like a great next token prediction thing because you’re kinda like on a path and like, it doesn’t really matter what you’ve done before.”
Speaker
Alessio Fanelli
Publisher
Latent Space

04 / belief

You’ve not mentioned hardware at all, like sensors or obviously we you mentioned you don’t do chips. I think even in AV there’s, like, a big, cameras versus lidars.

“You’ve not mentioned hardware at all, like sensors or obviously we you mentioned you don’t do chips. I think even in AV there’s, like, a big, cameras versus lidars.”
Speaker
Alessio Fanelli
Publisher
Latent Space

05 / belief

I remember initially, I think people put you and Scale AI very similarly for some things about being kinda like on the data infrastructure side of things.

“I remember initially, I think people put you and Scale AI very similarly for some things about being kinda like on the data infrastructure side of things.”
Speaker
Alessio Fanelli
Publisher
Latent Space

06 / belief

” I think a lot of founders are in a similar way where they wanna raise a lot of money to signal they’re strong, and you raise a lot of money without spending it.

“” I think a lot of founders are in a similar way where they wanna raise a lot of money to signal they’re strong, and you raise a lot of money without spending it.”
Speaker
Alessio Fanelli
Publisher
Latent Space

07 / uncertainty

We call that MD and agents that MD’s just the same how sim length. And so it is like, that works, but it feels like, yeah, I don’t know.

“We call that MD and agents that MD’s just the same how sim length. And so it is like, that works, but it feels like, yeah, I don’t know.”
Speaker
Alessio Fanelli
Publisher
Latent Space

08 / evaluation

The problem is that you need twice the amount of ram, twice the amount of, you know, it’s like, it’s kind of taxing on the machine.

“The problem is that you need twice the amount of ram, twice the amount of, you know, it’s like, it’s kind of taxing on the machine.”
Speaker
Alessio Fanelli
Publisher
Latent Space

10 / prediction

The price of like writing code is going to zero, blah, blah, blah. But it actually seems like the value of having some sort of platform substrate is like increasing because as you build these new things, you can kind of plug them together.

“The price of like writing code is going to zero, blah, blah, blah. But it actually seems like the value of having some sort of platform substrate is like increasing because as you build these new things, you can kind of plug them together.”
Speaker
Alessio Fanelli
Publisher
Latent Space

11 / belief

Definition of Turbo puffer, because I think you could be a Vector db, which is maybe a bad word now in some circles, you could be a search engine.

“Definition of Turbo puffer, because I think you could be a Vector db, which is maybe a bad word now in some circles, you could be a search engine.”
Speaker
Alessio Fanelli
Publisher
Latent Space

12 / belief

You know, I think sometimes it’s like, well, these are the tradeoffs, but the three nines, it’s like, actually it’s not a real trade off because we can make something that nobody has ever made before and actually make it work.

“You know, I think sometimes it’s like, well, these are the tradeoffs, but the three nines, it’s like, actually it’s not a real trade off because we can make something that nobody has ever made before and actually make it work.”
Speaker
Alessio Fanelli
Publisher
Latent Space

14 / belief

Can I do it with a hundred, if I need a trillion healthcare tokens, it’s like, they’re probably not out there that you don’t have, you know, I think that’s really like the.

“Can I do it with a hundred, if I need a trillion healthcare tokens, it’s like, they’re probably not out there that you don’t have, you know, I think that’s really like the.”
Speaker
Alessio Fanelli
Publisher
Latent Space

15 / belief

Doubling, tripling every year in size is not like, uh, you know, and I think today you kind of see that with LLMs too, where like every year the jumps in size and like capabilities are just so big.

“Doubling, tripling every year in size is not like, uh, you know, and I think today you kind of see that with LLMs too, where like every year the jumps in size and like capabilities are just so big.”
Speaker
Alessio Fanelli
Publisher
Latent Space

20 / belief

I think now you can start to infer semantics from things that are beyond just like simple recognition to like understanding why certain things are happening a certain way.

“I think now you can start to infer semantics from things that are beyond just like simple recognition to like understanding why certain things are happening a certain way.”
Speaker
Alessio Fanelli
Publisher
Latent Space

21 / recommendation

That, that's why I, I would say I search is the first example of one of the things we used to mention on, you know, we had X on the podcast and perplexity obviously as a, as an API.

“That, that's why I, I would say I search is the first example of one of the things we used to mention on, you know, we had X on the podcast and perplexity obviously as a, as an API.”
Speaker
Alessio Fanelli
Publisher
Latent Space

23 / preference

We try not to chase as many trends and I don't know, I, you know, I was a founder myself and sometimes I feel like it's easy to just jump in and do the thing that is hot, but like becoming a founder to do something that is like underappreciated or like doesn't yet work shows some level of like dread and self, like you, you actually really believe in the thing.

“We try not to chase as many trends and I don't know, I, you know, I was a founder myself and sometimes I feel like it's easy to just jump in and do the thing that is hot, but like becoming a founder to do something that is like underappreciated or like doesn't yet work shows some level of like dread and self, like you, you actually really believe in the thing.”
Speaker
Alessio Fanelli
Publisher
Latent Space

24 / belief

You know, in the, as anything changed, like since you, because you updated this in 2022 and I think now we're kind of like, you know, five years removed from COVID and all of that.

“You know, in the, as anything changed, like since you, because you updated this in 2022 and I think now we're kind of like, you know, five years removed from COVID and all of that.”
Speaker
Alessio Fanelli
Publisher
Latent Space

27 / preference

Sometimes I feel like there's no decision-making in some things like, uh, today I built a autosave for like our internal notes platform and I literally just ask them cursor.

“Sometimes I feel like there's no decision-making in some things like, uh, today I built a autosave for like our internal notes platform and I literally just ask them cursor.”
Speaker
Alessio Fanelli
Publisher
Latent Space

32 / belief

I've run a lot of engineering teams in the past, and I think the product versus engineering tension has always been more about effort than like whether or not the feature is buildable.

“I've run a lot of engineering teams in the past, and I think the product versus engineering tension has always been more about effort than like whether or not the feature is buildable.”
Speaker
Alessio Fanelli
Publisher
Latent Space

36 / preference

First of all, you have very good support for mocking in unit tests, which is something that a lot of other frameworks don't do. So, you know, my favorite Ruby library is VCR because it just, you know, it just lets me store the HTTP requests and replay them.

“First of all, you have very good support for mocking in unit tests, which is something that a lot of other frameworks don't do. So, you know, my favorite Ruby library is VCR because it just, you know, it just lets me store the HTTP requests and replay them.”
Speaker
Alessio Fanelli
Publisher
Latent Space

37 / belief

I think that's kind of like the, that's the, you know, Sam is always talking about incremental deployments and kind of like getting, having people getting used to it.

“I think that's kind of like the, that's the, you know, Sam is always talking about incremental deployments and kind of like getting, having people getting used to it.”
Speaker
Alessio Fanelli
Publisher
Latent Space

39 / belief

Before we go into the inference, some of the deeper stuff, can you give people an overview of like some of the numbers? So I think last I checked, you have like 1.

“Before we go into the inference, some of the deeper stuff, can you give people an overview of like some of the numbers? So I think last I checked, you have like 1.”
Speaker
Alessio Fanelli
Publisher
Latent Space

40 / uncertainty

Oh, okay. Yeah, that one I don't know. I'm curious, like, you know, it's kind of like similar content, but different platform.

“Oh, okay. Yeah, that one I don't know. I'm curious, like, you know, it's kind of like similar content, but different platform.”
Speaker
Alessio Fanelli
Publisher
Latent Space

41 / belief

Um, and so I think if you remove that whole, like, oh, that's impossible, and you just think really clearly about like, what's now possible with like what, what they've done with O1, it's easy to see how that scales.

“Um, and so I think if you remove that whole, like, oh, that's impossible, and you just think really clearly about like, what's now possible with like what, what they've done with O1, it's easy to see how that scales.”
Speaker
Alessio Fanelli
Publisher
Latent Space

42 / belief

I'm like, I'm sure there's not what event, you know, but I'm curious, like, just like how people there's always like this, I think for a little bit, it went away about like startups and kind of like hustle culture and like all of that.

“I'm like, I'm sure there's not what event, you know, but I'm curious, like, just like how people there's always like this, I think for a little bit, it went away about like startups and kind of like hustle culture and like all of that.”
Speaker
Alessio Fanelli
Publisher
Latent Space

45 / evaluation

Yeah. And the big thing was like the mixed trial price fights, you know, and I think now it's almost like there's nowhere to go, like, you know, Gemini Flash is like basically giving it away for free.

“Yeah. And the big thing was like the mixed trial price fights, you know, and I think now it's almost like there's nowhere to go, like, you know, Gemini Flash is like basically giving it away for free.”
Speaker
Alessio Fanelli
Publisher
Latent Space

46 / observation

Sombra to congregate, and then the AI engineer summit. And that's why when I look at our growth chart, it's kind of like a proxy for like the AI engineering industry as a whole, which is almost like, like, even if we don't do that much, we keep growing just because there's so many more AI engineers.

“Sombra to congregate, and then the AI engineer summit. And that's why when I look at our growth chart, it's kind of like a proxy for like the AI engineering industry as a whole, which is almost like, like, even if we don't do that much, we keep growing just because there's so many more AI engineers.”
Speaker
Alessio Fanelli
Publisher
Latent Space

47 / belief

I think in AI you see this a lot, which is like a lot of stars, a lot of interest at a rate that you didn't really see in the past in open source, where nobody's running to start.

“I think in AI you see this a lot, which is like a lot of stars, a lot of interest at a rate that you didn't really see in the past in open source, where nobody's running to start.”
Speaker
Alessio Fanelli
Publisher
Latent Space

53 / belief

I think the point of the talk was like everybody, we're scaling these chips, we're scaling the compute, but like the second ingredient which is data is not scaling at the same rate.

“I think the point of the talk was like everybody, we're scaling these chips, we're scaling the compute, but like the second ingredient which is data is not scaling at the same rate.”
Speaker
Alessio Fanelli
Publisher
Latent Space

58 / belief

I mean, I'm curious, I think Dylan, At the debate he said SweetBench 80 percent was like a soap for end of next year as a kind of like, you know, watermark that the moms are still improving.

“I mean, I'm curious, I think Dylan, At the debate he said SweetBench 80 percent was like a soap for end of next year as a kind of like, you know, watermark that the moms are still improving.”
Speaker
Alessio Fanelli
Publisher
Latent Space

60 / evaluation

Um, and I think today the, the problem is that, Yeah, the agents are, that most people are building are good at following instruction, but are not as good as like extracting them from you.

“Um, and I think today the, the problem is that, Yeah, the agents are, that most people are building are good at following instruction, but are not as good as like extracting them from you.”
Speaker
Alessio Fanelli
Publisher
Latent Space

62 / evaluation

I think to me that the most interesting is like rest and GraphQL is almost more interesting in the world of agents because agents could come up with so many different things to query versus like before I always thought GraphQL was kind of like not really necessary because like, you know what you need, just build the rest end point for it.

“I think to me that the most interesting is like rest and GraphQL is almost more interesting in the world of agents because agents could come up with so many different things to query versus like before I always thought GraphQL was kind of like not really necessary because like, you know what you need, just build the rest end point for it.”
Speaker
Alessio Fanelli
Publisher
Latent Space

68 / belief

How did you get, you joined Anthropic, did you already know you were going to work on of the stuff you publish or you kind of join and then you figure out where you land? I think people are always curious to learn more.

“How did you get, you joined Anthropic, did you already know you were going to work on of the stuff you publish or you kind of join and then you figure out where you land? I think people are always curious to learn more.”
Speaker
Alessio Fanelli
Publisher
Latent Space

74 / evaluation

And eventually, you know, I started doing venture six, five years ago. And I think just like so many people in Europe reach out and ask, hey, can you like talk to our team and they just cannot comprehend like the risk appetite that people have here.

“And eventually, you know, I started doing venture six, five years ago. And I think just like so many people in Europe reach out and ask, hey, can you like talk to our team and they just cannot comprehend like the risk appetite that people have here.”
Speaker
Alessio Fanelli
Publisher
Latent Space

75 / evaluation

When I tried to set up Slack, it was like, hey, give me access to all channels and everything, which for the average person probably makes sense because you don't want to re-prompt them every time you add new channels.

“When I tried to set up Slack, it was like, hey, give me access to all channels and everything, which for the average person probably makes sense because you don't want to re-prompt them every time you add new channels.”
Speaker
Alessio Fanelli
Publisher
Latent Space

76 / evaluation

I've written a post called Maximum Enterprise Utilization, kind of like you have MFU for GPUs, but it's basically like so many people are focused on, oh, it's going to like displace jobs and whatnot. But I'm like, there's so much work that people don't do because they don't have the people.

“I've written a post called Maximum Enterprise Utilization, kind of like you have MFU for GPUs, but it's basically like so many people are focused on, oh, it's going to like displace jobs and whatnot. But I'm like, there's so much work that people don't do because they don't have the people.”
Speaker
Alessio Fanelli
Publisher
Latent Space

78 / belief

I think the thing about ad-generated content, if we look at YouTube, like we do videos on YouTube and it's like, you know, a lot of people like screaming in the thumbnails to get clicks.

“I think the thing about ad-generated content, if we look at YouTube, like we do videos on YouTube and it's like, you know, a lot of people like screaming in the thumbnails to get clicks.”
Speaker
Alessio Fanelli
Publisher
Latent Space

80 / commitment

If you have a model that is being used to generate election-related content, you want to see where it runs, whether or not it's running in a safe environment. And obviously, there's more on the geopolitical side that we will not touch on.

“If you have a model that is being used to generate election-related content, you want to see where it runs, whether or not it's running in a safe environment. And obviously, there's more on the geopolitical side that we will not touch on.”
Speaker
Alessio Fanelli
Publisher
Latent Space

82 / observation

How do you think about the importance of storage and like, do you kind of feel storage is like almost solved, where it's like, hey, you can kind of store these files anywhere, what matters is like access.

“How do you think about the importance of storage and like, do you kind of feel storage is like almost solved, where it's like, hey, you can kind of store these files anywhere, what matters is like access.”
Speaker
Alessio Fanelli
Publisher
Latent Space

88 / evaluation

It's not, it's not what gives us our edge, but it certainly means that then we don't have to build it and maintain it afterwards. So, it's a really good first step, I think, in, like, the overall maturity of the fine tuning product and API in terms of where they're going to see those early products.

“It's not, it's not what gives us our edge, but it certainly means that then we don't have to build it and maintain it afterwards. So, it's a really good first step, I think, in, like, the overall maturity of the fine tuning product and API in terms of where they're going to see those early products.”
Speaker
Alessio Fanelli
Publisher
Latent Space

91 / belief

Then once you get to thought generation, people start to think, what is going on here? So I think everybody, well, not everybody, but people that were tweaking with these things early on saw the take a deep breath, and things step-by-step, and all these different techniques that the people had.

“Then once you get to thought generation, people start to think, what is going on here? So I think everybody, well, not everybody, but people that were tweaking with these things early on saw the take a deep breath, and things step-by-step, and all these different techniques that the people had.”
Speaker
Alessio Fanelli
Publisher
Latent Space

96 / belief

Especially for newer folks that have like a lot more training data out there, so to speak. I think of like, you know, Sean Carroll.

“Especially for newer folks that have like a lot more training data out there, so to speak. I think of like, you know, Sean Carroll.”
Speaker
Alessio Fanelli
Publisher
Latent Space

97 / uncertainty

You see it in the fan fiction world, you know, people just come out with new things about the same franchise, like Harry Potter, just to have more things to read. So, yeah, I'm curious what that does, especially to, uh, allowing new IP kind of to come up when you have like such as iteration of successful ones, but I don't know.

“You see it in the fan fiction world, you know, people just come out with new things about the same franchise, like Harry Potter, just to have more things to read. So, yeah, I'm curious what that does, especially to, uh, allowing new IP kind of to come up when you have like such as iteration of successful ones, but I don't know.”
Speaker
Alessio Fanelli
Publisher
Latent Space

98 / recommendation

I think that makes a lot of sense and we're still maybe in the, everybody wants something else that is not transformers, you know, uh, but maybe the, the lesson is to not, to not move away too much.

“I think that makes a lot of sense and we're still maybe in the, everybody wants something else that is not transformers, you know, uh, but maybe the, the lesson is to not, to not move away too much.”
Speaker
Alessio Fanelli
Publisher
Latent Space

99 / prediction

The v100 is about 130 teraflops of kind of like compute the gb200 at fp4 is like 20, 000 teraflops so the hardware alone today got much more powerful and I would love to maybe hear from you how at the time you were thinking about optimizing for the hardware today versus how much of an insight you had into the hardware that was coming especially working at NVIDIA and maybe people have the same discussion today it's like you know Should we optimize for the hardware of today or like for the hardware of tomorrow, because we need the results today, you know, as a business, but sometimes maybe we waste some time.

“The v100 is about 130 teraflops of kind of like compute the gb200 at fp4 is like 20, 000 teraflops so the hardware alone today got much more powerful and I would love to maybe hear from you how at the time you were thinking about optimizing for the hardware today versus how much of an insight you had into the hardware that was coming especially working at NVIDIA and maybe people have the same discussion today it's like you know Should we optimize for the hardware of today or like for the hardware of tomorrow, because we need the results today, you know, as a business, but sometimes maybe we waste some time.”
Speaker
Alessio Fanelli
Publisher
Latent Space

101 / belief

I think to me, the most interesting thing has been hiring and some of the awesome people that you've been bringing on that maybe don't fit the central casting of Silicon Valley, so to speak.

“I think to me, the most interesting thing has been hiring and some of the awesome people that you've been bringing on that maybe don't fit the central casting of Silicon Valley, so to speak.”
Speaker
Alessio Fanelli
Publisher
Latent Space

113 / belief

I think maybe a lot of people initially wrote them off, but between some of the Gemini Nano stuff, like Gemma 2, PolyGemma, we'll talk about some of the KV cache and context caching.

“I think maybe a lot of people initially wrote them off, but between some of the Gemini Nano stuff, like Gemma 2, PolyGemma, we'll talk about some of the KV cache and context caching.”
Speaker
Alessio Fanelli
Publisher
Latent Space

117 / prediction

I think now for the first time, there's a clear path to how do we make a 7b model good without having to go through GPT-4 or going to Cloud 3. And we'll kind of talk about this later, but I think we're seeing maybe the, not the death, but settling the picks and shovels, it's kind of going away.

“I think now for the first time, there's a clear path to how do we make a 7b model good without having to go through GPT-4 or going to Cloud 3. And we'll kind of talk about this later, but I think we're seeing maybe the, not the death, but settling the picks and shovels, it's kind of going away.”
Speaker
Alessio Fanelli
Publisher
Latent Space

120 / belief

If you think about the evolution of the models, I think up until Llama3, with Meta AI and some of these things, I'm like, it makes sense that they want to build their own models and they're multi-modal.

“If you think about the evolution of the models, I think up until Llama3, with Meta AI and some of these things, I'm like, it makes sense that they want to build their own models and they're multi-modal.”
Speaker
Alessio Fanelli
Publisher
Latent Space

122 / prediction

I think there's a lot of chatter obviously about synthetic data and like there was the Rephrase the Web paper that came out maybe a few months ago about using, you know, Mastral to make training data better.

“I think there's a lot of chatter obviously about synthetic data and like there was the Rephrase the Web paper that came out maybe a few months ago about using, you know, Mastral to make training data better.”
Speaker
Alessio Fanelli
Publisher
Latent Space

123 / evaluation

I think there was obviously the Kepler, and then there was Chinchilla, and then people kind of got the Llama scaling law, like the 100 to 200x parameter to token ratio.

“I think there was obviously the Kepler, and then there was Chinchilla, and then people kind of got the Llama scaling law, like the 100 to 200x parameter to token ratio.”
Speaker
Alessio Fanelli
Publisher
Latent Space

124 / evaluation

Are people just finding out recently about these problems because now the scores are getting so high that you're actually inspecting the benchmarks and maybe in the past you were scoring so badly that maybe you weren't as worried about the overall quality?

“Are people just finding out recently about these problems because now the scores are getting so high that you're actually inspecting the benchmarks and maybe in the past you were scoring so badly that maybe you weren't as worried about the overall quality?”
Speaker
Alessio Fanelli
Publisher
Latent Space

126 / observation

You know, people just trust the person saying the thing assertively, even though it's false, and then actually try and figure out what the truth is. So yeah, I think you mentioned that, you know, it's like a more social experiment.

“You know, people just trust the person saying the thing assertively, even though it's false, and then actually try and figure out what the truth is. So yeah, I think you mentioned that, you know, it's like a more social experiment.”
Speaker
Alessio Fanelli
Publisher
Latent Space

133 / belief

Any other things that you want to bring up, maybe how people are using gradient, anything like that, I think that will help have a clearer picture for people.

“Any other things that you want to bring up, maybe how people are using gradient, anything like that, I think that will help have a clearer picture for people.”
Speaker
Alessio Fanelli
Publisher
Latent Space

135 / belief

People are running the same thing when they're building sales agents, when they're building customer support agents, like it all comes down to how do you make the thing sound like how you want it to sound? And I think most folks out there do prompt engineering, but I feel like you figure out something that is much better than a good prompt.

“People are running the same thing when they're building sales agents, when they're building customer support agents, like it all comes down to how do you make the thing sound like how you want it to sound? And I think most folks out there do prompt engineering, but I feel like you figure out something that is much better than a good prompt.”
Speaker
Alessio Fanelli
Publisher
Latent Space

136 / evaluation

For example, with one system, there are like 780 endpoints. And if you're actually trying to do vector similarity, it's not that good because the people that wrote the specs didn't have in mind making them like semantically apart.

“For example, with one system, there are like 780 endpoints. And if you're actually trying to do vector similarity, it's not that good because the people that wrote the specs didn't have in mind making them like semantically apart.”
Speaker
Alessio Fanelli
Publisher
Latent Space

138 / belief

I think that's, that's like what I'm most interested to figure out because with the browsers, it's like, it's the entry point to the thing.

“I think that's, that's like what I'm most interested to figure out because with the browsers, it's like, it's the entry point to the thing.”
Speaker
Alessio Fanelli
Publisher
Latent Space

139 / evaluation

People were like, okay, this is better than this on this benchmark, blah, blah, blah, because maybe they did not have a lot of use cases that they did frequently.

“People were like, okay, this is better than this on this benchmark, blah, blah, blah, because maybe they did not have a lot of use cases that they did frequently.”
Speaker
Alessio Fanelli
Publisher
Latent Space

140 / belief

I think when you do a new Silicon, you're like, Oh, we're going to be so much better at this thing or like much faster, much cheaper.

“I think when you do a new Silicon, you're like, Oh, we're going to be so much better at this thing or like much faster, much cheaper.”
Speaker
Alessio Fanelli
Publisher
Latent Space

141 / uncertainty

I don't know. But again, it's just like the rumors are always floating around, you know but I think like, this is, you know, we're not going to get to the end of the year without Jupyter you know, that's definitely happening.

“I don't know. But again, it's just like the rumors are always floating around, you know but I think like, this is, you know, we're not going to get to the end of the year without Jupyter you know, that's definitely happening.”
Speaker
Alessio Fanelli
Publisher
Latent Space

142 / prediction

And Jensen, at his keynote, he did talk about synthetic data a little bit. So I think that's something that we'll definitely hear more and more of in the enterprise, which never bodes well, because then all the, all the people with the data are like, Oh, the enterprises want to pay now?

“And Jensen, at his keynote, he did talk about synthetic data a little bit. So I think that's something that we'll definitely hear more and more of in the enterprise, which never bodes well, because then all the, all the people with the data are like, Oh, the enterprises want to pay now?”
Speaker
Alessio Fanelli
Publisher
Latent Space

143 / prediction

Surface is a little smaller for image generation. So if you go back maybe six, nine months, most people will tell you, why would you build a coding assistant when like Copilot and GitHub are just going to win everything because they have the data and they have all the stuff.

“Surface is a little smaller for image generation. So if you go back maybe six, nine months, most people will tell you, why would you build a coding assistant when like Copilot and GitHub are just going to win everything because they have the data and they have all the stuff.”
Speaker
Alessio Fanelli
Publisher
Latent Space

144 / evaluation

Cloud is better. It's very good, you know, it's much better, it seems to me, it's much better than GPT 4 at doing writing that is more, you know, I don't know, it just got good vibes, you know, like the GPT 4 text, you can tell it's like GPT 4, you know, it's like, it always uses certain types of words and phrases and, you know, maybe it's just me because I've now done it for, you know, So, I've read like 75, 80 generations of these things next to each other.

“Cloud is better. It's very good, you know, it's much better, it seems to me, it's much better than GPT 4 at doing writing that is more, you know, I don't know, it just got good vibes, you know, like the GPT 4 text, you can tell it's like GPT 4, you know, it's like, it always uses certain types of words and phrases and, you know, maybe it's just me because I've now done it for, you know, So, I've read like 75, 80 generations of these things next to each other.”
Speaker
Alessio Fanelli
Publisher
Latent Space

145 / belief

I think like the, what is possible today and like what is worth investing in, you know? And I think like, I mean, people look at you and say, well, these guys are building agents.

“I think like the, what is possible today and like what is worth investing in, you know? And I think like, I mean, people look at you and say, well, these guys are building agents.”
Speaker
Alessio Fanelli
Publisher
Latent Space

149 / evaluation

I think before we wrap, you have written a blog post that can show about good hearts law impact in ML, which is, you know, when you measure something, then the thing that you measure is not a good metric anymore because people optimize for it.

“I think before we wrap, you have written a blog post that can show about good hearts law impact in ML, which is, you know, when you measure something, then the thing that you measure is not a good metric anymore because people optimize for it.”
Speaker
Alessio Fanelli
Publisher
Latent Space

150 / belief

I think to me the most interesting thing about long inference is like, You're shifting the cost to the customer depending on how much they care about the end result.

“I think to me the most interesting thing about long inference is like, You're shifting the cost to the customer depending on how much they care about the end result.”
Speaker
Alessio Fanelli
Publisher
Latent Space

153 / uncertainty

I was gonna say, maybe it's not here, but I don't know if we want to talk about diffusion transformers as like in the alt architectures, just because of Zora.

“I was gonna say, maybe it's not here, but I don't know if we want to talk about diffusion transformers as like in the alt architectures, just because of Zora.”
Speaker
Alessio Fanelli
Publisher
Latent Space

154 / belief

You know I think, I mean, obviously the Voyager paper is like the most basic example where like, now their artifact is like the best planning to do a diamond pickaxe in Minecraft.

“You know I think, I mean, obviously the Voyager paper is like the most basic example where like, now their artifact is like the best planning to do a diamond pickaxe in Minecraft.”
Speaker
Alessio Fanelli
Publisher
Latent Space

155 / belief

I think the The challenge, well not the challenge, what they need to figure out is like how do you keep the rag piece always up to date constantly, you know, I feel like the models, you put all this work into pre training them, but then at least you have a fixed artifact.

“I think the The challenge, well not the challenge, what they need to figure out is like how do you keep the rag piece always up to date constantly, you know, I feel like the models, you put all this work into pre training them, but then at least you have a fixed artifact.”
Speaker
Alessio Fanelli
Publisher
Latent Space

156 / evaluation

I think if anything, RAG's complexity goes up and up the more you use it, you know, because you have more data sources, more things you want to put in there.

“I think if anything, RAG's complexity goes up and up the more you use it, you know, because you have more data sources, more things you want to put in there.”
Speaker
Alessio Fanelli
Publisher
Latent Space

158 / evaluation

I think in Europe, I walked through a lot of the posters and whatnot, there seems to be mode collapse in a way in the research, a lot of people working on the same things.

“I think in Europe, I walked through a lot of the posters and whatnot, there seems to be mode collapse in a way in the research, a lot of people working on the same things.”
Speaker
Alessio Fanelli
Publisher
Latent Space

161 / belief

Like this is a concept that I hadn't heard before reading about this. So I think most people's mental models, like transformers or something else, it’s not transformers AND something else.

“Like this is a concept that I hadn't heard before reading about this. So I think most people's mental models, like transformers or something else, it’s not transformers AND something else.”
Speaker
Alessio Fanelli
Publisher
Latent Space

163 / evaluation

I know we kind of binned the lightning round in the last few episodes, but I think for you two, one of the questions we used to ask is like, what's the most interesting unsolved question in AI?

“I know we kind of binned the lightning round in the last few episodes, but I think for you two, one of the questions we used to ask is like, what's the most interesting unsolved question in AI?”
Speaker
Alessio Fanelli
Publisher
Latent Space

165 / belief

When you were building retool vectors, how do you think about, yeah, leveraging a startup to do it, putting vectors into one of the existing data stores that you already had? I think like, you're really a quite large customer scale.

“When you were building retool vectors, how do you think about, yeah, leveraging a startup to do it, putting vectors into one of the existing data stores that you already had? I think like, you're really a quite large customer scale.”
Speaker
Alessio Fanelli
Publisher
Latent Space

168 / belief

Obviously, there's like all these different things like privacy, you know, if an internal tool hallucinates, that's fine, because you're paying people to use it basically, versus if it hallucinates to your customer, there's a different bar. Because for you, if people build internal tool with retool, there are external customers to you, you know, so I think you're on the flip side of it.

“Obviously, there's like all these different things like privacy, you know, if an internal tool hallucinates, that's fine, because you're paying people to use it basically, versus if it hallucinates to your customer, there's a different bar. Because for you, if people build internal tool with retool, there are external customers to you, you know, so I think you're on the flip side of it.”
Speaker
Alessio Fanelli
Publisher
Latent Space

171 / belief

I think the other interesting thing that you have is this inline assist UX that I wouldn't say async, but like it works while you can also do work.

“I think the other interesting thing that you have is this inline assist UX that I wouldn't say async, but like it works while you can also do work.”
Speaker
Alessio Fanelli
Publisher
Latent Space

174 / belief

Uh, yeah, I think one of the things that Swyx said when he was opening the AI engineer summit a couple of weeks ago was like, look, most people here don't know much about the space because it's so new and like being open and welcoming.

“Uh, yeah, I think one of the things that Swyx said when he was opening the AI engineer summit a couple of weeks ago was like, look, most people here don't know much about the space because it's so new and like being open and welcoming.”
Speaker
Alessio Fanelli
Publisher
Latent Space

176 / belief

I think the startup's point is the most interesting, right? You mentioned companies that are GPU poor, they raise a lot of money, and there's a lot of startups out there that are GPU poor and did not raise a lot of money.

“I think the startup's point is the most interesting, right? You mentioned companies that are GPU poor, they raise a lot of money, and there's a lot of startups out there that are GPU poor and did not raise a lot of money.”
Speaker
Alessio Fanelli
Publisher
Latent Space

177 / evaluation

I think, to me, the biggest takeaway was like and I was talking with Mike Conover, another friend of the podcast, about this is they're kind of staying in the single threaded, like, synchronous use cases lane, you know?

“I think, to me, the biggest takeaway was like and I was talking with Mike Conover, another friend of the podcast, about this is they're kind of staying in the single threaded, like, synchronous use cases lane, you know?”
Speaker
Alessio Fanelli
Publisher
Latent Space

178 / belief

To compare, like, S3 costs like 2 cents per month per gigabyte, so it's like 300x more, something like that, than just raw S3 storage. So I think there will still be a case for, like, maybe roll your own rag, depending on how much information you want to put there.

“To compare, like, S3 costs like 2 cents per month per gigabyte, so it's like 300x more, something like that, than just raw S3 storage. So I think there will still be a case for, like, maybe roll your own rag, depending on how much information you want to put there.”
Speaker
Alessio Fanelli
Publisher
Latent Space

179 / belief

I think that would be quite, quite interesting. But yeah, I think a lot of excitement and, you know, we just announced the, the Linux based launchpad, so we're on the side of the, of the builders.

“I think that would be quite, quite interesting. But yeah, I think a lot of excitement and, you know, we just announced the, the Linux based launchpad, so we're on the side of the, of the builders.”
Speaker
Alessio Fanelli
Publisher
Latent Space

183 / belief

You know, the data is not good for anything. And I think like now, at least with these models, they have some knowledge of their own and they can also tell you if your data is bad, which I think is like something that before you didn't have.

“You know, the data is not good for anything. And I think like now, at least with these models, they have some knowledge of their own and they can also tell you if your data is bad, which I think is like something that before you didn't have.”
Speaker
Alessio Fanelli
Publisher
Latent Space

184 / belief

Now that's an important message. And yeah, that's why we've been promoting a lot of open source developers, open source communities, I think, letting the builders build and explore.

“Now that's an important message. And yeah, that's why we've been promoting a lot of open source developers, open source communities, I think, letting the builders build and explore.”
Speaker
Alessio Fanelli
Publisher
Latent Space

185 / evaluation

So I think the easiest thing for people to grasp so far has been Mojo, which is a superset of Python. And I think everybody talks about that because it's easier to grasp, but Modular's goal is to build a unified AI engine.

“So I think the easiest thing for people to grasp so far has been Mojo, which is a superset of Python. And I think everybody talks about that because it's easier to grasp, but Modular's goal is to build a unified AI engine.”
Speaker
Alessio Fanelli
Publisher
Latent Space

186 / evaluation

I think in one of your previous podcasts, you mentioned leaving people behind, you know, that are like not experts in certain things and they can't contribute.

“I think in one of your previous podcasts, you mentioned leaving people behind, you know, that are like not experts in certain things and they can't contribute.”
Speaker
Alessio Fanelli
Publisher
Latent Space

187 / belief

The difference I think at the Databricks keynote, you said chains are like predetermined steps and agents is models reasoning to figure out what steps to take and what actions to take.

“The difference I think at the Databricks keynote, you said chains are like predetermined steps and agents is models reasoning to figure out what steps to take and what actions to take.”
Speaker
Alessio Fanelli
Publisher
Latent Space

188 / uncertainty

There was actually my first article on the Twilio blog with a Python script to like predict pricing of like Daily Fantasy players based on my past week performance. Yeah, I don't know.

“There was actually my first article on the Twilio blog with a Python script to like predict pricing of like Daily Fantasy players based on my past week performance. Yeah, I don't know.”
Speaker
Alessio Fanelli
Publisher
Latent Space

190 / preference

But anyway, yeah, curious to see. And I think another thing from your Twitter model parades that I really like is actually differentiating between the type of workload.

“But anyway, yeah, curious to see. And I think another thing from your Twitter model parades that I really like is actually differentiating between the type of workload.”
Speaker
Alessio Fanelli
Publisher
Latent Space

193 / belief

I think a lot of times the focus is on tokens parameter ratio in the training dataset and people don't think as much about the actual flops per GPU, which you're going to mention later in the blog post too, in terms of how much you can get out.

“I think a lot of times the focus is on tokens parameter ratio in the training dataset and people don't think as much about the actual flops per GPU, which you're going to mention later in the blog post too, in terms of how much you can get out.”
Speaker
Alessio Fanelli
Publisher
Latent Space

194 / evaluation

If you have a eight bits model quantized down, you need one byte per parameter. So for example, in an H100, which is 80 gigabyte of memory, you could fit a 70 billion parameters in eight, you cannot fit a FP32 because you will need like 280 gigabytes of memory.

“If you have a eight bits model quantized down, you need one byte per parameter. So for example, in an H100, which is 80 gigabyte of memory, you could fit a 70 billion parameters in eight, you cannot fit a FP32 because you will need like 280 gigabytes of memory.”
Speaker
Alessio Fanelli
Publisher
Latent Space

195 / prediction

I think after all of this, you can quickly do the math and see that training needs to be distributed to actually work because we just don't have hardware that can easily run this.

“I think after all of this, you can quickly do the math and see that training needs to be distributed to actually work because we just don't have hardware that can easily run this.”
Speaker
Alessio Fanelli
Publisher
Latent Space

200 / commitment

Because I think one of the big moments with Uber pajama was like, okay, we can take the LAMA one data mixture, use all the open source data sets and just run GPUs at them.

“Because I think one of the big moments with Uber pajama was like, okay, we can take the LAMA one data mixture, use all the open source data sets and just run GPUs at them.”
Speaker
Alessio Fanelli
Publisher
Latent Space

202 / belief

I think you, right now the math was that for the price of the most expensive thing we build, which is the International Space Station, we could build one Tampa of.

“I think you, right now the math was that for the price of the most expensive thing we build, which is the International Space Station, we could build one Tampa of.”
Speaker
Alessio Fanelli
Publisher
Latent Space

204 / evaluation

I think I've talked about this on the podcast, but this idea of like just-in-time UIs, you know, like each type of user wants to interact in a different way.

“I think I've talked about this on the podcast, but this idea of like just-in-time UIs, you know, like each type of user wants to interact in a different way.”
Speaker
Alessio Fanelli
Publisher
Latent Space

205 / uncertainty

I think when you first tweeted about this, I don't know if you already accepted the job, but you tweeted about this, and then the next one was like, this is a NotionAI subtweet.

“I think when you first tweeted about this, I don't know if you already accepted the job, but you tweeted about this, and then the next one was like, this is a NotionAI subtweet.”
Speaker
Alessio Fanelli
Publisher
Latent Space

209 / evaluation

I think I talked about it on the podcast before, but like the switch from syntax to like semantics, like developers used to be focused on the syntax and not the meaning of what they're writing.

“I think I talked about it on the podcast before, but like the switch from syntax to like semantics, like developers used to be focused on the syntax and not the meaning of what they're writing.”
Speaker
Alessio Fanelli
Publisher
Latent Space

210 / belief

You know, we mentioned some of the podcasts you had done, Jonathan, I think in one of them you mentioned Mosaic was not planning on building a model and releasing and obviously you eventually did.

“You know, we mentioned some of the podcasts you had done, Jonathan, I think in one of them you mentioned Mosaic was not planning on building a model and releasing and obviously you eventually did.”
Speaker
Alessio Fanelli
Publisher
Latent Space

211 / belief

I think fast training in inference was like one of the goals, right? So there's always the trade off between doing the hardest thing and like.

“I think fast training in inference was like one of the goals, right? So there's always the trade off between doing the hardest thing and like.”
Speaker
Alessio Fanelli
Publisher
Latent Space

214 / evaluation

I'm building an agent internally for us. And Guardrails are obviously very exciting because once you set the initial prompt, like the model creates its own prompts.

“I'm building an agent internally for us. And Guardrails are obviously very exciting because once you set the initial prompt, like the model creates its own prompts.”
Speaker
Alessio Fanelli
Publisher
Latent Space

215 / prediction

But good luck with the rest of your fundraise. But it's like, never mention a fundraise, but because in the prompt, it, as part of the prompt is like, if it's a pitch and it's not in the space, a pre-draft, an email, it thinks it has to do it a lot more than it should.

“But good luck with the rest of your fundraise. But it's like, never mention a fundraise, but because in the prompt, it, as part of the prompt is like, if it's a pitch and it's not in the space, a pre-draft, an email, it thinks it has to do it a lot more than it should.”
Speaker
Alessio Fanelli
Publisher
Latent Space

216 / evaluation

Yeah. This is super interesting because right now a lot of products are kind of the same because all I do is they call it the model and some are prompted a little differently, but you can only guess so much delta between them in the future.

“Yeah. This is super interesting because right now a lot of products are kind of the same because all I do is they call it the model and some are prompted a little differently, but you can only guess so much delta between them in the future.”
Speaker
Alessio Fanelli
Publisher
Latent Space

217 / recommendation

Yeah. So if I were a founder today, I wouldn't worry as much about the model, maybe, but I would say, okay, what can I build into my product and like, or what can I do at the engineering level that maybe it's not model optimization because everybody's working on it, but like you said, it's like, why haven't people thought of this before?

“Yeah. So if I were a founder today, I wouldn't worry as much about the model, maybe, but I would say, okay, what can I build into my product and like, or what can I do at the engineering level that maybe it's not model optimization because everybody's working on it, but like you said, it's like, why haven't people thought of this before?”
Speaker
Alessio Fanelli
Publisher
Latent Space

219 / evaluation

The question is, if you wanna compete against these companies, maybe the model is not what you're gonna do it with because the open source kind of commoditizes it.

“The question is, if you wanna compete against these companies, maybe the model is not what you're gonna do it with because the open source kind of commoditizes it.”
Speaker
Alessio Fanelli
Publisher
Latent Space

222 / belief

How do you see the engineer, I, I think Sean, you said copilot is everybody gets their own junior engineer to like write some of the code and then you fix it For me, a lot of it is the junior engineer gets a senior engineer to actually help them write better code.

“How do you see the engineer, I, I think Sean, you said copilot is everybody gets their own junior engineer to like write some of the code and then you fix it For me, a lot of it is the junior engineer gets a senior engineer to actually help them write better code.”
Speaker
Alessio Fanelli
Publisher
Latent Space

223 / belief

Zero to like a million users in five days and everybody, I, I think there's like dozens of ChatGPT API wrappers on GitHub that are unofficial and clearly people want the product.

“Zero to like a million users in five days and everybody, I, I think there's like dozens of ChatGPT API wrappers on GitHub that are unofficial and clearly people want the product.”
Speaker
Alessio Fanelli
Publisher
Latent Space
Search evidence