High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works

Public evidence record

Jonathan Frankle

Published podcast speaker

Claims
21
Episodes
2
Shows
1
Named items
0

Claim ledger

What Jonathan said.

3 transcript-backed records

01 / evaluation

I spend a lot of time thinking about RAG these days. And RAG, you know, in one sense, you know, the way that I think about RAG is it's the world's simplest agent.

“I spend a lot of time thinking about RAG these days. And RAG, you know, in one sense, you know, the way that I think about RAG is it's the world's simplest agent.”
Publisher
Latent Space

02 / evaluation

And I, I do think the vibe based eval cannot be, you know, underrated really at the, I mean, at the end of the day we, we did stop our models and do vibe checks and we did, as we monitor our models, one of our evals was we just had a bunch of prompts and we would watch the answers as the model trained and see if they changed cuz honestly, You know, I don't really believe in any of these eval metrics to capture what we care about.

“And I, I do think the vibe based eval cannot be, you know, underrated really at the, I mean, at the end of the day we, we did stop our models and do vibe checks and we did, as we monitor our models, one of our evals was we just had a bunch of prompts and we would watch the answers as the model trained and see if they changed cuz honestly, You know, I don't really believe in any of these eval metrics to capture what we care about.”
Publisher
Latent Space

03 / evaluation

Um, boy inference, that's a whole second thing, and a whole different animal from training. And at the end of the day, when we founded the company, our belief was that inference was relatively well served at that time.

“Um, boy inference, that's a whole second thing, and a whole different animal from training. And at the end of the day, when we founded the company, our belief was that inference was relatively well served at that time.”
Publisher
Latent Space
Search evidence