High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works
← All source episodes

Latent Space / episode intelligence

The Winds of AI Winter (Q2 Four Wars Recap) + ChatGPT Voice Mode Preview

2 Aug 2024 52 published claims 3 attributable people
Open original episode ↗ Read the evidence

Speakers in the public record

Shawn WangAlessio FanelliEthan Sutin

Claim mix

belief 30uncertainty 9commitment 4preference 3recommendation 2evaluation 2prediction 2

Evidence policy

Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.

Claim ledger

The useful parts, with receipts.

52 published records

08 / belief

Open source is synonymous with local, private, and all the good things that people want. And I think their vision of even running this stuff on CPUs at a very, very fast speed by just being extremely cracked, I think is very understated.

“Open source is synonymous with local, private, and all the good things that people want. And I think their vision of even running this stuff on CPUs at a very, very fast speed by just being extremely cracked, I think is very understated.”
Speaker
Shawn Wang
Publisher
Latent Space

17 / belief

I think that you know, the fact that it could simulate reverberations was a little interesting There's just so many things that obviously are kind of, like, not available now, but it's capable of.

“I think that you know, the fact that it could simulate reverberations was a little interesting There's just so many things that obviously are kind of, like, not available now, but it's capable of.”
Speaker
Ethan Sutin
Publisher
Latent Space

23 / belief

We just didn't have the paper beforehand. But I think, like, when I call Lama3, the synthetic data model is you have the license for it, but then you also have the roadmap, the recipe, because it's in the paper.

“We just didn't have the paper beforehand. But I think, like, when I call Lama3, the synthetic data model is you have the license for it, but then you also have the roadmap, the recipe, because it's in the paper.”
Speaker
Shawn Wang
Publisher
Latent Space

27 / belief

To me, I think a lot of people were trying to puzzle out the strategic moves between OpenAI and Apple here because Apple is in a very good position to commoditize OpenAI.

“To me, I think a lot of people were trying to puzzle out the strategic moves between OpenAI and Apple here because Apple is in a very good position to commoditize OpenAI.”
Speaker
Shawn Wang
Publisher
Latent Space

28 / belief

I think maybe a lot of people initially wrote them off, but between some of the Gemini Nano stuff, like Gemma 2, PolyGemma, we'll talk about some of the KV cache and context caching.

“I think maybe a lot of people initially wrote them off, but between some of the Gemini Nano stuff, like Gemma 2, PolyGemma, we'll talk about some of the KV cache and context caching.”
Speaker
Alessio Fanelli
Publisher
Latent Space

30 / belief

Some things don't change. But on top of that, I think one of the reasons I put vector databases inside of these wars is in order to grow, the vector databases have to become more frameworks.

“Some things don't change. But on top of that, I think one of the reasons I put vector databases inside of these wars is in order to grow, the vector databases have to become more frameworks.”
Speaker
Shawn Wang
Publisher
Latent Space

32 / uncertainty

I mean, it's interesting what it can really, what it can really nail, and then where it kind of like like you said, kind of mixes it up, or like, blends it, and I, I don't know, it just seems intuitional wise that like, the longer the session is, and I don't know exactly how, You know, like, once you have a message history of, like, a bunch of audio tokens.

“I mean, it's interesting what it can really, what it can really nail, and then where it kind of like like you said, kind of mixes it up, or like, blends it, and I, I don't know, it just seems intuitional wise that like, the longer the session is, and I don't know exactly how, You know, like, once you have a message history of, like, a bunch of audio tokens.”
Speaker
Ethan Sutin
Publisher
Latent Space

34 / commitment

I will call out, you know, they have some interesting experimentations with Mamba and Mistral NEMO is actually on the efficiency frontier chart that I drew that is still relevant.

“I will call out, you know, they have some interesting experimentations with Mamba and Mistral NEMO is actually on the efficiency frontier chart that I drew that is still relevant.”
Speaker
Shawn Wang
Publisher
Latent Space

43 / preference

I think I want to leave some discussion time open for miscellaneous trends that are happening in the industry that don't exactly fit in the four words or are a layer above the four words.

“I think I want to leave some discussion time open for miscellaneous trends that are happening in the industry that don't exactly fit in the four words or are a layer above the four words.”
Speaker
Shawn Wang
Publisher
Latent Space

46 / evaluation

These are the same things to the model. That is a huge, huge win for interpretability, because up to now, we were only doing interpretability on toy models, like a few million parameters, a model of Go or chess or whatever.

“These are the same things to the model. That is a huge, huge win for interpretability, because up to now, we were only doing interpretability on toy models, like a few million parameters, a model of Go or chess or whatever.”
Speaker
Shawn Wang
Publisher
Latent Space

47 / prediction

I think now for the first time, there's a clear path to how do we make a 7b model good without having to go through GPT-4 or going to Cloud 3. And we'll kind of talk about this later, but I think we're seeing maybe the, not the death, but settling the picks and shovels, it's kind of going away.

“I think now for the first time, there's a clear path to how do we make a 7b model good without having to go through GPT-4 or going to Cloud 3. And we'll kind of talk about this later, but I think we're seeing maybe the, not the death, but settling the picks and shovels, it's kind of going away.”
Speaker
Alessio Fanelli
Publisher
Latent Space

48 / recommendation

We should basically, when we're getting to the point where we're over-training models 100 times past Chinchilla ratio to optimize for inference, the next thing is actually like, hey, let's stop using so much memory when training because we're going to quantize it anyway for inference.

“We should basically, when we're getting to the point where we're over-training models 100 times past Chinchilla ratio to optimize for inference, the next thing is actually like, hey, let's stop using so much memory when training because we're going to quantize it anyway for inference.”
Speaker
Shawn Wang
Publisher
Latent Space

49 / prediction

They will be building on architectures like Mobile LM and Small LM, which basically innovate in terms of shared weights and shared matrices for small models so that you just optimize the amount of file size and memory that you take up. And I think just general trend on device models, the only way that intelligence too cheap to meter happens is everything happens on device.

“They will be building on architectures like Mobile LM and Small LM, which basically innovate in terms of shared weights and shared matrices for small models so that you just optimize the amount of file size and memory that you take up. And I think just general trend on device models, the only way that intelligence too cheap to meter happens is everything happens on device.”
Speaker
Shawn Wang
Publisher
Latent Space

50 / uncertainty

Because I think that one of the burning rumors is, I don't know, nothing from Scott, I haven't talked to him at all about this, even though he's very friendly.

“Because I think that one of the burning rumors is, I don't know, nothing from Scott, I haven't talked to him at all about this, even though he's very friendly.”
Speaker
Shawn Wang
Publisher
Latent Space
Search evidence