Speaker unverified: commitment

Transcript context

…re. And we didn't want lose that because it was one of the things we did well and underestimated. And my takeaway is actually we do it better than other experiences. So we certainly want to keep that. So what we did was now we're just updating their hypothesis to achieve the same goal, which is these things are working and when people want to discover, they use them and they seem to work, they can also get better. Gustav Söderström You're on this hill-climbing journey from a machine learning point of view, but the question is, how do you make sure that whenever people feel that they are in that I'm trapped in my taste bubble, they understand that these things are there and they're easy to use? So now we have a version of Home that we are also testing, obviously, where these things are very available but voluntary and you can still do all of the recall. And so from my point of view, this is the reason we A/B test because you want to be scientific about it and you want to learn as much as possible about your own product and your users. And now I'm sharing a lot of the learnings. Maybe we should keep them to ourselves, but my hunch is that it's going to make it a much better product. Gustav Söderström But what I told my teams when we went into this, because I've done this a few times, agree to signing, I think there are two fundamentally different types of product development. One is designing a new feature. It is hard, but it's voluntary for people to use. So you do the AI DJ. Some people love it, that's fine. If you don't like it, it didn't make it worse for you. But when you redesign, it is much more tricky because it's not voluntary to participate in the redesign. So there's a cost even for people who don't like it. Then you have a very tricky problem here, which is there are going to be two types of feedback. One is you did something and it was right, but people are upset because you changed stuff. The other is you did something and it wasn't right, and people are also upset but for good reasons. Gustav Söderström And so how do you separate these two? Because I think I explained this to ... When we talk through this with my teams, I think the analogy to think about is you have your desktop, your physical desktop, you have your computer in one place, you have your pencil over here, you have your notebook over there, and I come in and I just rearrange all of it. And you have spent, in our case, maybe 12 years with that setup. It doesn't matter if I have a lot of quantitative data that my new setup is better, you're going to get upset because you are effective in this old setup. And it's hard to tell those apart. The most classic use case is the Facebook newsfeed, which people are very upset about when it became a single newsfeed. But it turned out to solve a lot of user problems that you didn't have to run around all of Facebook collecting events yourself. Gustav Söderström So there are some ways of understanding if you made it better, but people's habits are broken or if it's not better. And one thing is, for example, to look at new user cohorts that don't have that behavior versus all user cohorts and so forth. you made it better, but people's habits are broken or if it's not better. And one thing is, for example, to look at new user cohorts that don't have that behavior versus all user cohorts and so forth. So we went through all of this with the teams. Before we did it, I said, "This is going to be painful." There's probably going to be a lot of tweets because chances that we get it exactly right are very low. So for that reason, it hasn't been very hard on the team. It is hard ... You want to respond to people, but the right way to do it is to listen, understand, try new hypothesis to really figure out what's going on. So I think I've done it maybe three or four times now. Three maybe. One unsuccessful, two successfully. So kind of knew what I was getting into. Gustav Söderström So it's almost like you punish yourself, very painful, but also the most exciting things. And I think any product person knows that the easiest, the most straightforward thing to do is to iterate around where you are. There's no risk. You're not going to get fired, no user is going to get angry. But everyone also knows that eventually if you don't adapt new technologies, new paradigms, et cetera, you're going to get replaced. You have to find this balance of trying new things. And when you work in software, you have this tool of A/B testing and being scientific about it. When you build hardware, it's worse. If you're wrong, you're wrong. You can't update. I love this story. I so appreciate you sharing it. I imagine also with a big launch like this, you can't actually A/B test it ahead of time because of the press season. They're like, "Oh my God, look what Spotify's doing." And so you're limited there. Imagine, right? You couldn't really test this ahead of time.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.