Tom Verrilli: evaluation

Transcript context

…Yeah. Okay. I'm going to take us to a recurring corner of the podcast, Fail Corner. Why I like doing this is because people see people like you coming on the podcast, have this illustrious career, everything's going great constantly, you're just killing it. And they don't see the things that don't work out and the times that you failed. And in their life, things often go wrong. So the question for you is just what's an example of a time in your career where things failed, something you built, some career move you made that didn't work out, and then what'd you learn from that experience? Honestly, mate, and it's very nice of you to say nice things, but I feel like I fail more often than I succeed across the course of my career. Genuinely, to the blog you referenced right at the start, I published in the back of that the actual document we use internally to talk about how we build. And it starts with batting .500 is the goal. So you're hoping to be right as often as you're wrong. So there's probably just too many specific examples of times I've screwed up in my career, but there probably is a really common thread to it. And I think it's probably an easy trap for any PM to fall into, which is averages mean nothing to the individual is probably the thing that I've really scarred by. In any sizable population, it's really attractive to go and look at average utility or average adoption of something. And then you find that only 3% of people use something. And you're like, "Cool, we can probably get rid of that feature. It's not used widely." But if you don't go a layer deeper, and be like, actually it's only 3% of something, but there's a group of people for whom it's 100% of what they do. This is their core use case. And for expediency's sake, because somebody doesn't want to maintain a feature anymore, you're just going to deprecate it. And then it turns out you blow up the use case of that group of humans. And then to your last point about network effects, the ongoing spiral effect of that can be enormous. I think about it a lot in e-commerce of this is somebody's business. If we're just not reliable or we're deprecating a feature, it's kind of like a Westfield Mall just turning off the power in the lead up to Christmas without thinking about it. And so there are real downstream impacts to people's businesses that often come from just a lack of nuance in understanding metrics, particularly averages. They just lie to you all the time. And I think I've probably screwed up in all of the ways in my career, but most of the time I've made genuinely I'm disappointed in myself levels of decisions, it's typically that I've relied on averages without thinking about the individual use cases that are hidden underneath. Makes me think about Jeff Bezos has a quote, "When you have data and an anecdote, trust the anecdote."…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.