Annie Duke: evaluation

Transcript context

…And here's nevertheless, exactly. Fair enough. I think we're going to start having to pivot this podcast to a parenting podcast because there's so much stuff here, but I'm going to try to resist. So we've been talking about decision-making and frameworks and things like that. Something I wanted to ask you is how much better can someone get at making better decisions? So for example, say someone listens to all of your podcasts, reads your book, studies it intensely. What's the delta you find in somebody being able to make a better decision? And where [inaudible 00:20:35] comes from is Daniel Kahneman of all people. People ask him, "Do you just live such a optimal life now that you know all of these biases we have and all these mental errors we make?" And he's like, "No, I can't..." I think he said famously, "I don't, actually. I can't use these in practice. This is just stuff that I have learned, but it doesn't actually impact my day to day." So I'm curious just how much better can someone get? What kind of delta have you found in terms of making decisions? Okay, so it depends on whether you actually do the things. I mean, I think that that's what the issue is. I think Kahneman did some work way back that started him on this journey. That was basically work on hiring and it was taking it from completely unstructured Lenny, saying, "I just know a great product manager when I see one", whatever that means, to me going to Lenny and saying, "Okay, I understand that you think that you see a great product manager you know one when you see one", but can you explain to me what that means? In the abstract, what are the things that you're looking for in someone that you want to fill that role? And we can then excavate that, right? And make what is implicit because you're applying some implicit model to how you're thinking about the person that you're interviewing and how they map onto the role that they're going to be, but let's make that explicit. So we can make that explicit, we can turn it into a decision rubric, we can create a structured interview process out of that. And then what he found was after you've gone through that process, if you then use your intuition after having done that, not before, that then you actually can really drastically improve your hit rate on hiring. And in that case, it was from about a 50% hit rate to 65. So that's pretty huge, so that's a really big difference. The problem is nobody does it. So I can tell you that most of the conversations that I had with Danny that were about my work in particular were just him saying, "How do you get anybody to do it?" Now that's not to say that I would be better at that than him, it's that he didn't do my job, right? So he was an academic doing research and so on and so forth, I'm living in companies embedding for years. I've been with one client for five years, one client for four years, one client for four years also, almost four years, three and a half years. And then I just took on a new client who I've been with for six months, and that's my whole client roster because I stay with them so long. So if you do it, the answer is quite a bit. Like at least in terms of Danny's work in relation to something pretty noisy like hiring, you went from 50% hit rate to 65% hit rate, which is huge. It's enormous. The problem is that the way that you can make decisions in order to be better at them is not supernatural to the way that humans make decisions. I think we think a lot more highly of our intuition than we really ought to. We think very highly of our ability to notice things in the moment and act rationally toward them in a way that really we ought not to. We tend to think that we have insight that other people don't have when actually the other people probably have more insight into our situation than we do. So I think the answer is a lot with a big if, with an if you can get people to do it. And I think that's where the issue comes in. ther people probably have more insight into our situation than we do. So I think the answer is a lot with a big if, with an if you can get people to do it. And I think that's where the issue comes in. Now, the good news is that you can then reverse that and say if I'm willing to do it, think about what an edge I'm going to have over people who aren't willing to do it, right? So we can think there's people who don't know about it, there's people who do know about it but don't do it, and then there's people who do know about it, but do do it. And it's that last group that's so tiny. So the answer is I think a lot, but nobody does it.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.