High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works
← All source episodes

Latent Space / episode intelligence

2024 in Synthetic Data and Smol Models [LS Live @ NeurIPS]

24 Dec 2024 5 published claims 0 attributable people
Open original episode ↗ Read the evidence

Speakers in the public record

Claim mix

belief 2observation 1prediction 1preference 1

Evidence policy

Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.

Claim ledger

The useful parts, with receipts.

5 published records

01 / observation

LLAMA was trained on 1 trillion tokens, but LLAMA3 8b was trained on 15 trillion tokens. So Meta managed to get a model that's the same size, but But it performs so much [00:20:00] better by choosing to like spend the sacrifice during training, because as we know, training is a one time cost, but inference is something that's ongoing.

“LLAMA was trained on 1 trillion tokens, but LLAMA3 8b was trained on 15 trillion tokens. So Meta managed to get a model that's the same size, but But it performs so much [00:20:00] better by choosing to like spend the sacrifice during training, because as we know, training is a one time cost, but inference is something that's ongoing.”
Speaker
Not verified from transcript
Publisher
Latent Space

02 / prediction

I think if you do that approach, it's normal to [00:05:00] observe this kind of behavior because the quality is going to be worse because the model is already small.

“I think if you do that approach, it's normal to [00:05:00] observe this kind of behavior because the quality is going to be worse because the model is already small.”
Speaker
Not verified from transcript
Publisher
Latent Space

04 / belief

I think that like in AI, we just started like with fine tuning, for example, trying to make BERT work on some specific use cases, and really struggling to do that.

“I think that like in AI, we just started like with fine tuning, for example, trying to make BERT work on some specific use cases, and really struggling to do that.”
Speaker
Not verified from transcript
Publisher
Latent Space
Search evidence