Speaker unverified: evaluation

Transcript context

…w summers ago," or "Here's some new stuff that were trending yesterday in The Last of Us episode or something like that." Gustav Söderström So that product couldn't have existed without generative AI, both generating the voice and generating the content of what the voice says. So you can have individualized, personalized voice at the scale of half a billion people. And so we had the use case we have seen for many, many years. Sometimes people call it the radio use case. We called it the zero intent use case internally when you actually don't know what you want to listen to at all. Gustav Söderström Spotify wasn't that good. Spotify was good when, at least roughly, you knew the use case of what you want to do, if it was a workout or dinner. We had lots of options for all of those. But if you really didn't know at all, it was hard to open Spotify and stare at it. And people used to say longingly that this was the one thing that radio was good at. Radio was quite bad, to be honest. I mean, it's not personalized to you at all. It's not on demand. You come in in the middle of things, it's actually terrible in many ways. But people still often say that there was something good about it. And I think that's something was the fact that you had a knob and you could just switch between contexts. It's like no, boring, boring, boring, boring, okay, this is good. Gustav Söderström And Spotify never had that mode of, I don't know what I want, but I want to cycle through things until I find something that I like. And I think with the AI DJ, that's actually the use case we managed to solve. So X comes on and says, "I'm going to suggest something to you that you can listen to." And if you like it, you can keep listening, but if you don't like it, you bring him back again and you change channel. And for one reason or another, we tried to solve that for many times for a long time, but just starting to play a random song without any context as to why you would hear this, it just never worked. So that was our first foray into a product that couldn't exist before. And I think to your question of principles around that, there are a few pretty distinct principles that we've learned. Gustav Söderström One that I really like that is not my principle at all, I think it is straight from Chris Dixon, is the principle of fault-tolerant user interfaces. So I can't say how many times during the early machine learning era when we said we're moving from curation to recommendation. I saw a design sketch that was a single big play button because clearly that is the simplest user interface you can do, but if you don't understand the performance of your machine learning, you can't design for it. The quality of your machine learning, if you're going to have a single play button, needs to be literally 100% or zero prediction error, and that's never the case. So let's say that you have a one in five hits, four out of five things are done, then you need a UI that probably at least shows five things at the same time on screen. So you have a one in five of something being relevant on screen. in five hits, four out of five things are done, then you need a UI that probably at least shows five things at the same time on screen. So you have a one in five of something being relevant on screen. Gustav Söderström So you need to understand the performance of your machine learning to design for it. It needs to be fault tolerant and often you need an escape hatch for the user. So you make a prediction. But if you were wrong, it needs to be super easy for the user say, "No, you're wrong, I want to go to my library or to this or to that." So we have that principle of having fault-tolerant user interface and a user interface that corresponds to the current performance of your algorithms. And I think that is going to be true for generative machine learning as well. I think a very clear example actually is Mid Journey. If you think about the early Mid Journey user interface inside the Discord channel, actually generating an image was very, very slow. Gustav Söderström It took a long time to generate high-quality image and they could have built the silver button thing where you put in a prompt, you wait for minutes, you get an image, and I think one out of four times, it's going to be bad. So you would've been disappointed three out of four times and it's a minute each, so like four minutes later, you'd be, "This is a shitty product." What they did was they generated four simultaneous low-res images very quickly and you could say, "So apparently, their performance was probably one in four, that's why they showed four and not six." And so one in four was usually pretty good. You click that one and either continue to iterate or scale it up. So that's also an example of, I think, people understanding where the performance of generative AI was when they built the UI. So that's something that I would be inspired by. Gustav Söderström And for the AI DJ specifically, another principle is to try to avoid this urge of just wanting to show off the technology and then have this voice that talk and talk and talk and talk. You have to remember that people came there for the music. So the principle for the AI DJ coming from the team, by the way, this was a bottoms-up product actually, it required a lot of support. We actually acquired big companies and so forth to be able to build it. But the idea had been built by teams bottom up. So the principle there was literally to do as little as possible and get out of the way. And I think that was really helpful. It's not telling you what the weather is and what happened in the news and going on and on and on about this band. It is trying to get you to the music and I think that's why it's working because it is working very well for us. I love this distinction between recommendation and generation. And this begs the question of, there's this trend that I imagine you're seeing of people autogenerating music using artists catalog. There's this Drake and The Weeknd thing that came out a week or two ago. Where do you think this ends up going and how do you think artists adjust to this world where music can just be autogenerated? This play button is all of it is generated versus just like the DJ in between the songs.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.