Lenny Rachitsky: belief

Transcript context

…That's true. That's true. Though some people do joke that if we don't treat our agents nice, what's going to happen when AGI comes? And, you know, maybe it still might benefit us to be kind. I'm one of those people that says thank you to the Waymo when I leave and just like thanks ChatGPT when I'm in voice mode, just like thank you, that was really helpful. So along these lines, I know there's a lot of ways to go here, but just in terms of skills that are important to a manager, Which do you think are most valuable to develop in working with agents in AI systems? I think about things like clarity communication, just like what comes to mind when you think about like, here's the things you want to double down on as you're learning to be manager that will also help you be really good at AI tooling and working with agents. The first is defining the goal and defining the outcome and being really, really crystal clear on what does success look like. I mean, there's obviously lots and lots of, like if you ask a company to do this, we'll know that this is challenging for humans, right? I think a lot of times when you talk about why is alignment so difficult at a big company, it often comes down to this question, which is, Different people may have different pictures of what success looks like. And even if I describe in human words, oh, you know, Lenny, I want to build this product and it's going to be amazing. Or this podcast episode, which you asked me, well, lots of people to hear it and take away things. That's very general. Like, how do we get even more specific so that we know without question whether we've hit it or not? This is actually a really, really difficult problem. It's a difficult question for us because again, we tend to think very high level. So figuring out how to boil it down so that an agent can really understand what success and failure looks like is a lot of the game. And I think this also relates to things like, well, that's why we have to write evals. And that's why they're so important because they're helping us understand what is the objective criteria. And these days I work in data and my company is all about trying to automate data analysis. And the forever question goes, the whole point of data and the whole point of metrics and KPIs is we're trying to put a little bit more of an objective measure or get as crystal clear as possible about what success looks like. And I think it's really an art more than it is like a science, but that's like the first thing. I think if you're really unclear about what success looks like, the prompt, you're probably not going to get the most amazing work. I think that's true for managing teams and it's very much true for managing AIs.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.