Luke Drago: preference

Transcript context

…It is really one of the big kind of technical, perhaps even political questions of our time. We have AI models that are aligned to certain interests. There's a whole separate question of whether we can even align them to certain interests. And that, in my opinion, is an unsolved problem. But they happen to have certain goals, certain preferences. And those preferences are a kind of a mix of what the companies are interested in, what governments are interested in, and what end users are interested in. And the balance between which preferences should be strongest in the model, that is a very interesting question and something that, I think there's a lot of work to be done there. For example, I say in not that long, I expect us to have personal agents that can do our e-mail and our calendar for us. that agent, is that agent working on my behalf when I ask it to book a hotel for me? Or is there perhaps a kind of a corporate preference to book a certain hotel that OpenAI might have an agreement with, something like that? You could quite easily see the incentives of the model or the preferences of the model becoming modeled between what the end user wants and what the companies are interested in. Do you see a principled way to solve this? Or is this just like any other product where the company selling the product is interested in something and the consumer is interested in somewhat of the same thing, but the preference sets do not perfectly overlap? I think if you talk to a model and you ask it for something, it should do one of two things. It should either answer in your interest or tell you when it's not. If we're going to go down the kind of rabbit hole of LLM monetization via advertisement, it should be exceptionally clear what is an advertisement, what isn't. I think it was started this way in search, and it's less so now. But even still, if you're searching something on Google and you type in something, you can see which things are ads. This should be really obvious, because of course, I happen to believe in what we're building is that these things should be loyal to your interests, that OpenAI or Anthropic or us, that we shouldn't sign some sort of a deal and then disguise or nefariously let you know, hey, by the way, have this hotel you should be looking at. It's a really bad situation to be in if your model doesn't work for you. And I think that this is just true as a consumer. You want to know that when you are asking something for advice, you are getting the kinds of advice, the kinds of information, the kinds of truth that you would give to a friend because you genuinely care about them. It's what makes these tools useful, is that they work on your behalf. Imagine if, you know, there's a Black Mirror episode recently that really stuck with me. It was in the new season where like a woman has a brain transplant and they upload like half of her brain to the cloud. And this is great because she's still alive. But like every couple of hours, she turns off and gives a advertising pitch about something. She has no recollection of giving the advertising pitch, and then she wakes back up and doesn't even know what's happened. And she only finds this out because other people tell her, hey, why did you just bring up this travel site in the middle of your lecture? And it's great that the technology has enabled her to do this really cool thing. She's still alive. She's able to live her life, except, of course, if she suddenly needs to give a sponsored ad on something where she goes out of the coverage area. And because they have this monopoly control over her, because you don't have competing vendors for your brain upload. You've got like half your brain here, half of the processing power in the cloud. Only one guy's got that chip. And so what ends up happening is they start her on a very cheap plane. It's only a few 100 bucks a month. It's so good to be alive. And then they say, oh, we have this deluxe plan now. And you can go outside the coverage area if you buy the deluxe plan. And it's like, oh, you're now on our like freemium tier. And if you just upgrade a little bit more, you can get rid of the advertisements. And suddenly, the thing that made your life so much better is now a massive hindrance to your quality of life because one guy has total control and gets to jack up the rents as they see fit. That is the kind of scenario that we're trying to avoid. much better is now a massive hindrance to your quality of life because one guy has total control and gets to jack up the rents as they see fit. That is the kind of scenario that we're trying to avoid. Part of this comes through democratization, the technology. Part of this comes through ensuring that they're actually loyal to you. And my expectation here is in the future, if we get to the good future, everyone has got an agent that's aligned to them that advocates on their interests that they know is working for them. One thing I'll add here to close it up, to kind of close the loop here is that one of the places I really agree with Sam Altman on is his concept of AI privilege. The idea that actually, if you're giving this much information to a system, It probably shouldn't be used against you, and this is different than other technologies, so I'm probably someone who'd advocate for more privileging technologies rather than less, even on the status quo ones, but if you are constantly interacting with this thing and it's helping organize your life. That's a powerful tool in the hands of someone who wants to be nefarious to you, who wants to understand your life, who wants to interrogate it instead of you. And because it's a chatbot, you know, it's not going to know when it should reserve it. Maybe it could, but maybe it doesn't know when it should like, you know, use its Fifth Amendment rights. Not clear, it doesn't have the Fifth Amendment right now. It probably doesn't have a right against incriminating you. And if it has that much access to your life, it probably should. That's one of the more, I think, like really value-aligned things OpenAI's called for recently is some sort of concept like that, and I endorse that wholeheartedly.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.