Annie Duke: evaluation

Transcript context

…So the data is pretty strong that by the time you quit, it's probably long after the [inaudible 01:11:37]. And it's really just because, look, when we start things, we're starting things under difficult circumstances, which come from the uncertainty of the decision to start something. So when we start something, luck is going to have an influence on the outcome, which obviously, we have no control over because luck isn't in our control. And then there's also hidden information. So what happens is that after the fact, we know that we're going to learn new information and it can make it very hard to start things because we want to be more sure than we actually need to be. It's why Bezos has the 70% role to try to roll people back and be willing to accept that uncertainty in the starting decision. Now the good news is that when you learn that new stuff and the new stuff that you learn is, "Ooh, if I had known this, I wouldn't have started it", you have the option to quit generally. So that's the good news. The bad news is that the same difficulties that apply to the decision to start apply to the decision to stop. In other words, we're making that decision under uncertainty as well. So we're not going to know for sure whether it would've turned out well or poorly unless we continue to do the thing that we already started. And we don't like to walk away from things unless we know for sure. So as Richard Thaler put it, most people won't quit until it actually isn't a decision. In other words, the whole thing is blown up, the startup has no money, or you're up on Everest and the blizzard is literally upon you and you're stuck in it. Or I think as he said, until you've fallen in the crevasse already, then you'll make the decision to quit because then you know how it was really bad. So people generally, for example, don't quit their jobs until they feel they have no other choice or relationships or projects or products that they're developing, it all applies because we want to know for sure. And then on top of that is the fact that there is this issue of sunk cost, which is when we walk away, we feel like we'll have wasted everything that we've already put into what we're doing. But of course, waste is a prospective problem, not a retrospective one, even if we treat it like a retrospective one because it's the prospective one. Well, if you wouldn't start this today, then that means that everything that you're putting into this going forward is the actual waste, right? But we do that all the time. We go forward with things that we ought not to be going forward with because we're trying to protect the resources that we've already sunk into it in the past. Then there's other issues that have to do with, for example, endowment, the ownership over the things that we've built. This is particularly bad in product because we're building things. And once we build things, we own them. that have to do with, for example, endowment, the ownership over the things that we've built. This is particularly bad in product because we're building things. And once we build things, we own them. And once we own them, we don't want to give them up and we actually value them more highly than identical things that we don't own. And then there's issues of internal and external validity, which is really just a fancy way to say your identity. How do other people view you? How do you view yourself? Do you feel like you failed? And what that means is that by the time, there's so many biases against stopping that by the time you're actually even thinking about quitting, it's probably already past the time that you ought to have quit. But we'll still continue on until we know for sure we didn't have any choice because here's the thing. When you walk away from something and someone's like, "Hey, why'd you stop that?" And you're like, "Oh, I had no choice" and you tell them everything that went wrong and, nobody's going to question you. They're going to be like, "Oh, well, it sounds like you put in your best effort." But if you walk away early, people are like, "What?" So just quickly, I'll tell you I think just one of the best stories of this that I've got in my pocket here. Let me pull it out before the end. So Stewart Butterfield creating a product called Glitch, and Glitch is a massive multiplayer online world-building cooperative game. Releases it, this is in the aughts, and it's like a huge hit with the critics. It's Monty Python meets Dr. Seuss, it's an incredible whatever. They're getting tons and tons of great word of mouth and PR not doing any paid marketing. They have incredible investors in Andreessen Horowitz and Accel, they have $6 million in the bank and they 5,000 have diehard users, meaning users who use the game who play over 20 hours a week. The issue is that customer acquisition was a beast that for every one person who was playing over 20 hours a week, there were between 95 and 99 people who came for five minutes and left. So obviously, this is a customer acquisition problem, which everybody knows. So they make an agreement in 2012 that they're going to do paid marketing, which they do for six weeks. And during that six weeks, their acquisition of new users, it's growing like six to 7% week over week, which is amazing. And at the end of that six weeks, this is November of 2012, at the end of that six weeks, that Monday morning, Stewart Butterfield writes a note to his investors and co-founders and says, "I woke up this morning with the dead certainty that Glitch was over." Now notice nobody would do this in this case, right? But this is what happened is that the issue is that you really have to see is this worthwhile or not? Would I start this today? That's a forecast of the future, right? And what he did was some back-of-the-envelope math, and he said, "Look, if we continue to acquire customers at the weight that we've been acquiring them at the cost that we've been acquiring them, it's going to be 31 weeks until we break even." But that's an absurd assumption because customer acquisition costs, it's going to go up. been acquiring them at the cost that we've been acquiring them, it's going to be 31 weeks until we break even." But that's an absurd assumption because customer acquisition costs, it's going to go up. Cap has to rise because we're going to saturate the core gaming market, so it's got to rise. So what he realized at that point was that this was not a venture scale business, and he was in this for a venture scale business. So even though nobody else saw that he was supposed to shut it down, he saw he was supposed to shut it down. And not only that, he saw that he was supposed to shut it down for his employees who were working for equity, and he had now realized that the equity wasn't worth their time and that it wasn't fair to them to keep going with it. So he shuts it down. Obviously, that feels... Who does that? Right? And he will actually tell you that he knows he should have shut it down before the marketing push, but he needed the marketing push to prove it to himself that he was seeing the future clearly. Now, the code to this story just quickly is two days later, he's like, "Well, I'm a startup guy. I want to start something", and he's got this internal communication tool that his team is using in order to develop this product that everybody loves. And he says, "Actually, they really like that. Maybe that should be the next product." So he goes and talks to the investors, they roll their money over into that. And that thing, which had no name at the time, now gets a name which is searchable log of all company knowledge, which is Slack. And so I think this is the important thing to realize is that we get so focused on, "But what about everything that I've put into it?" When what we forget is that when you're doing something, there's not just the cost of doing something that's not worthwhile that's direct, but there's also the cost of not being able to devote your attention to other opportunities that might be available to you. And as smart as Stewart Butterfield is, he couldn't see Slack until he quit Glitch. And that is a true cost that he would've born of continuing with Glitch. If he had continued with Glitch, Slack would not be something that we're all using today.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.