Lenny's Podcast / episode intelligence
Speakers in the public record
Claim mix
Evidence policy
Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.
Claim ledger
23 published records
01 / recommendation
“So we recommend building step-by-step. When you start small, it forces you to think about what is the problem that I'm going to solve.”
02 / evaluation
“I mean, if you think of it's also the most beautiful part of AI, which is, I mean, we are all much more comfortable talking than following a bunch of buttons and all of that. So the bar to using AI products is much lower because you can be as natural as you would be with humans, but that's also the problem, which is there are tons of ways we communicate and you want to make sure that that intent is rightly communicated and the right actions are taken because most of your systems are deterministic and you want to achieve a deterministic outcome, but with non-deterministic technology and that's where it gets a little messy.”
03 / evaluation
“And there's so many other benefits to limiting autonomy because there's just danger also of the thing doing too much for you and just messing up your, I don't know, your database, sending out all these emails you never expected.”
04 / evaluation
“For instance, MRIs and block tests, because as soon as you know patient's information, it's easier to approve that and AI could do that versus something like an invasive surgery, et cetera, is more high risk.”
05 / belief
“I just saw Demis from DeepMind, AI, Google, whatever they call the whole org, talking about this where he thinks that's going to be a big part of where they're going, combining the image model work, the LLM, and also their world model stuff, Genie, I think is what it's called.”
06 / belief
“I would say I am extremely privileged in terms of working with really smart people in great companies in the Silicon Valley.”
07 / belief
“I'm not saying they're not important, but I think just this, I'm going to keep jumping across tools and going to pick up and learn if new tool is overrated.”
08 / belief
“I think with Kiriti, he's pretty calm and very grounded and he's always been my sounding board.”
09 / commitment
“In terms of what is going on in the community, I feel there's just this false dichotomy of this either evals is going to solve everything or online monitoring or production monitoring is going to solve everything. And I find no reason to trust one of the extremes in the sense that I will entirely bank my application on this or like that to solve the thing.”
10 / belief
“Just one more line of questioning here that I think that's on my mind is the reason this became kind of a big debate is Cloud Code.”
11 / belief
“Until now, I think LLMs have been our most commonly used modules, but as humans, we are multimodal creatures, I would say.”
12 / belief
“I think be foolish enough to believe that you can do anything if you put your heart to it, especially now because you have so much data at your hand that could be pointing towards the fact that you probably will be unsuccessful.”
13 / evaluation
“Successful companies right now building in any new area, they are going through the pain of learning this, implementing this and understanding what works and what doesn't work.”
14 / prediction
“The product has kind of been built so that your intention can be converted into a particular action and you kind of are clicking through a bunch of buttons, options, forms, and all of that, and you finally achieve your intention. But now that layer in AI products has completely been replaced by a very fluid interface, which is mostly natural language, which means the user can literally come up with a ton of ways of saying or communicating their intentions.”
15 / prediction
“Imagine you are a company who has a lot of customer support tickets and why even imagine OpenAI is the exact same thing when we were launching products and there was a huge spike of support volume as we launched successful products like Image or GPT-5 and things like that.”
16 / observation
“I want to take two steps back and kind of talk about how much weight the term evals has had to take in the second half of 2025 because you go meet a data labeling company and they tell you our experts are writing evals and then you have all of these folks saying that PMs should be writing evals, they're the new PRDs.”
17 / evaluation
“We're still so busy building AI products that we're not worried about security, but it will be such a huge problem to kind of, especially with this non-deterministic API again. So you're kind of stuck because there are tons of instructions that you could inject within your prompt and then it's going really bad.”
18 / recommendation
“What that means is constrain the number of decisions your AI systems can make and make sure that they're humans in the loop and then increase that over time because you're kind of building a flywheel of behavior and you're understanding what kind of use cases are coming in or how your users are using the system.”
19 / observation
“Recently, I think Air Canada had this thing where one of their agents predicted or hallucinated a policy for a refund, which was not part of their original playbook, and they had to go by it because legal stuff.”
20 / recommendation
“I think we did cover a bunch of best practices for AI products, which is start small, try to get your iteration going well and build a flywheel and all of that.”
21 / preference
“For me, it's Whisper Flow. I think I've been using it quite a bit and I didn't know I needed it so much.”
22 / evaluation
“The second one is the culture itself. And again, I work with enterprises where AI is not their main thing and they need to bring in AI into their processes just because a competitor is doing it.”
23 / preference
“I think to add on to that, I feel like persistence is also something that is extremely valuable, especially given that anybody who wants to build something, the information is at your fingertips even more than the past decade.”