High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works

Public evidence record

Hamel Husain

Published podcast speaker

Claims
9
Episodes
1
Shows
1
Named items
1

Books, apps, and tools

The evidenced stack.

Browse the grouped index →

app / likes

Claude Code

“Yeah, I really like Claude Code and I like it because I feel like the UX is outstanding.”

Lenny's Podcast · 25 Sept 2025

Evidence receipt · Source ↗

Claim ledger

What Hamel said.

9 transcript-backed records

02 / evaluation

If you look at the eval products, let's say the ones up until recently that some of the big labs have, they don't have error analysis. They have a suite of generic tools, cosine similarity, hallucination score, whatever, and that doesn't work.

“If you look at the eval products, let's say the ones up until recently that some of the big labs have, they don't have error analysis. They have a suite of generic tools, cosine similarity, hallucination score, whatever, and that doesn't work.”
Speaker
Hamel Husain
Publisher
Lenny's Podcast

04 / belief

Now, people say the word "Evals," trying to carve out this new thing, and saying evals and then A-B testing, but if you zoom out, it's the same data science as before, and I think that's what's causing the confusion is, "Hey, we need data science thinking," and AI product is helpful to have that thinking in AI products like it is in any product is my take on that.

“Now, people say the word "Evals," trying to carve out this new thing, and saying evals and then A-B testing, but if you zoom out, it's the same data science as before, and I think that's what's causing the confusion is, "Hey, we need data science thinking," and AI product is helpful to have that thinking in AI products like it is in any product is my take on that.”
Speaker
Hamel Husain
Publisher
Lenny's Podcast

05 / belief

" You could see the messiness of the real world in here, and the assistant just calls a tool that says transfer call, but it doesn't say anything. It just abruptly does transfer call, so it's pretty jank, I would say.

“" You could see the messiness of the real world in here, and the assistant just calls a tool that says transfer call, but it doesn't say anything. It just abruptly does transfer call, so it's pretty jank, I would say.”
Speaker
Hamel Husain
Publisher
Lenny's Podcast

06 / evaluation

Can't the AI just eval it?" That's the most common misconception, and people want that so much that people do sell it, but it doesn't work.

“Can't the AI just eval it?" That's the most common misconception, and people want that so much that people do sell it, but it doesn't work.”
Speaker
Hamel Husain
Publisher
Lenny's Podcast

07 / evaluation

There's a common trap that a lot of people fall into because they jump straight to the test like, "Let me write some tests," and usually that's not what you want to do.

“There's a common trap that a lot of people fall into because they jump straight to the test like, "Let me write some tests," and usually that's not what you want to do.”
Speaker
Hamel Husain
Publisher
Lenny's Podcast

08 / evaluation

" I'm like, "Yeah, let's look at it right now." They're surprised that I am going to go look at individual traces, and it always 100% of the time learn a lot and figure out what the problem is.

“" I'm like, "Yeah, let's look at it right now." They're surprised that I am going to go look at individual traces, and it always 100% of the time learn a lot and figure out what the problem is.”
Speaker
Hamel Husain
Publisher
Lenny's Podcast
Search evidence