Julie Zhuo: evaluation

Transcript context

…Okay, so I want to come back to this really interesting, unusual path that you took from being a head of design at Facebook. You're an inspiration to so many designers. Now, you spend your time on a data startup, obsessed with data. I don't know, classically, designers aren't the biggest fans of experiments and data and making decisions based on data. When you look at designers and hear designers kind of push back, I'm like, no, we don't want to be super data-driven. We want to we know better than we have a sense of what's beautiful and great and intuition, all these things. What do you think designers are missing when they feel that and say that when they're afraid of writing experiments and data and kind of want to push that out? There's one phrase that my co-founder and I would always discuss with us amongst ourselves very early on and which we shared with like a lot of the companies that we work with, which is what you really want is you want to diagnose with data and treat with design. So data is not a tool that's gonna tell you what you should build or what the solution is or how we're gonna cure the fact that you don't have really great retention. It's just not, but it can tell you if you have a problem and where that problem or opportunity might be, but you still need to go back and undergo a very creative process to figure out what's the best way to solve that. So that's the first thing I would say is this framework of like data helps you figure out what's actually happening, what do people like, what are they engaging with, whatnot, right? It's just, it gives you a story that better reflects reality. Because again, we all have stories, right? We're like, oh, my company is amazing. People love us, blah, blah. That's a story I want to believe, but reality may be a different picture. And so what data is trying to do is capture reality. And by the way, I don't think of data just as like it's A-B test and it's quantitative things we can measure. It's, to me, data is also well, what did people put onto TikTok and which things went viral and what are they saying in the Twitterverse or Xverse, I guess is what's called now. And if you do a customer interview, that's still all data. It's just that that is a little harder to understand distill and quantify, although now with AI, you know, we have better tools for synthesizing. So that's all data in my mind. And it's just all trying to help us understand what is really happening. What is the phenomenon that's happening in reality and how do we understand it? You still have to go and invent and create and dream. And there's no formula and there's no science that will tell you exactly how you're going to make a hit. You can experiment, which has allows you to try more things maybe and more rigorously understand what that does in the short term. They're going to have to, it's all very contextualized, right? Because AV tests don't tell you what will happen in the very long run. And that's still, again, it's all still data. You still have to synthesize and figure out what to do. So that's this thing I'll say, diagnose with data and treat with design. I think the second thing I will usually tell designers about is I find that sometimes maybe it's the, let's call it the false precision. of numbers that we kind of fall into right because it's like okay we got these numbers and the numbers go up it's like no the fact that you still have to choose which things you look at is an art not a science and your interpretation of if the number went up five percent is that good is that not good is also an interpretation and is an is an art not a science it's just that sometimes i think we can give ourselves this feeling. And I get it. number went up five percent is that good is that not good is also an interpretation and is an is an art not a science it's just that sometimes i think we can give ourselves this feeling. And I get it. Like sometimes there's this instinct to want to control things and we want everything to be buttoned up. And we want to know that if we did ABC, everything will be great. Our career is going to be awesome. Our product's going to rocket ship. And I think designers are rightly often pushing back and saying, no, the reality is like this stuff is ambiguous and there's uncertainty and we can never know for sure. And I think all that is quite true. So the other thing I would say that I really support is like you just actually can't make a really great product by thinking you can A, B test your way into it. So that's, I fundamentally believe that, but I don't think we should throw the baby out with the bathwater. I think there's actually, you know, these, it's not either, or it's not like data or design. It's like, these are just tools for us to use. And I would say every amazing designer that I've ever met is absolutely obsessed with trying to get a better understanding of reality. They want to know what users really think. They want to know what they're really doing. If they could read every user's minds, that's like the thing we would all want, right? As a designer is like, if I could just know what everyone is thinking and feeling every time they used it, my life would be a lot easier because then I would be able to build better and better things. And so that's what it's trying to help us do. It isn't perfect. No metric is going to tell you whatever we hope that it can in terms of the true certainty and precision. But it doesn't mean we can't use it to better our product development.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.