Marty Cagan: evaluation

Transcript context

…Makes me think about, men always joke that they're like, "Oh, I want to wear something more interesting to events. I always wear a suit. It's always got to be a suit. It's so boring." And other folks are like, "Shut up. Why would you want to complicate the life of men and having to dress more creatively? We got a good thing going with suits." And it makes me think about if the jobs of a PM are just to find the problems like, "All right. Let's go with that. Life's good." And turns out it's not. Anyway, your last point also made me think about this recent tweet by, I think it was Patrick Collison or John Collison, about how a lot of people think user research, it's like user research informs what you build. That's how people think about it. It's like user research leads to what you build. And his point is really interesting that user research informs your model of the user and the customer and that model should inform what you build. And you're constantly trying to refine this model, but you should have a model of your customer and your user in your head that you can come back to versus relying on user research to answer all your questions. What's your take on that perspective? Yeah. I mean, first of all, what they've done with Stripe is so awesome and that's another great example and I love what they picked and choose from great companies to create a culture for themselves. So I'd absolutely agree, but I'd go a little further. Because user research is a great topic. I mean, right there, it's a great product topic. I love it. I'm a big fanboy of user research. Well you just heard me basically arguing the same thing, don't be going out there spending all your time just validating the problem. Especially when we know, he's saying that too. He's trying to talk about building the mental model of our users, which is so important. Well designed products feed off of that. However, there's another layer around user research that I find is even a bigger source of confusion. In fact, I was just talking to a team this morning. That was saying that, yeah, what they do with user research is they test their prototypes. And when they get enough users saying how much they like it, they build it. And I'm like, "No, that's not why we do user research." And that is not going to fool any smart leaders. And now again, we're back to the problem discovery solution discovery, but most of our time needs to be on solution. And that's prototyping, that's testing with users, testing with customers, testing with stakeholders, testing with our developers. We're testing constantly. We are not just trying to find out if they like it. In fact, it's just the opposite. When we're doing user research, we're finding all the reasons they don't like it. In fact, that's an Elon Musk quote is when you do user a research, you should be focused on finding all the reasons they won't use your product. Even though Elon Musk has some issues right now, he's pretty good at product. And so he is very good at this. And that's how you want to think about that. The user researchers will talk about the difference between generative and evaluative user research. And so most of it, in terms of number of valuable things we get out of it, is evaluative. They are telling us the reasons they won't use it. The only other thing I'll add to that, you didn't ask about this, but it comes up all the time. I hate it when the user researchers go off and do the research themselves and bring back a report, not because they don't know what they're doing, they do know what they're doing. The reason is because the report is too often ignored. And so to me, the rule is, and I tell this to user researchers at the companies that I coach, if the product manager and the designer are not available to be there during their products test, cancel the test. They need to be there. This is what makes them useful to their team. I 100% agree. I have this memory of going to Paris with a research team, our head of Eng, head of design on our team, at least, and I and the researcher all went to Paris to do these focus groups with AirBnb hosts. And our researcher was very adamant that we come with her, that it's not just her coming back with tons of insights. And so 100% find value there. And engineers especially being involved in that process, I find to be super important.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.