Luke Drago: preference

Transcript context

…Spreading AI capabilities, that seems to me, when I read the kind of founding essays of OpenAI, that seems to be the vision that they had. They wanted to make sure that Google didn't have a monopoly on AI technology, and they wanted to empower everyone with AI models. And that vision seems to have kind of degraded over time. How do you make sure that doesn't happen to the vision you have for Workshop Labs? It is one of the things I think about the most, because the road to hell is paved with good intentions. It is paved with people who are working on things that ultimately end up working against their cause. Now, there's a couple of things here. I mean, there's the basic legal stuff. Like we're a public benefit corp with a fiduciary mission not to automate people. It's in lawyer speak for like enhancing economic opportunity, but that is like explicitly our goal. And this is instead of saying like, you kind of can do the generic thing of like to make sure AI benefits people. And it's like, okay, but what does that mean? Does that mean like we're going to put it in charge and then we think it's going to benefit people? Or does that mean we are going to try to do a certain thing. And in our case, this is this economic empowerment argument. It is our mission to make sure that AI actually meaningfully increases your power in the economy rather than decreasing it. I think also I'm A believer that personnel is policy. And so the kinds of people that you bring under the team will push you in certain directions. Our hiring process is laser focused on mission alignment, and it helps that we have been incredibly public. We kind of stumbled on this company on accident. We had worked on a bunch of research in the area quite publicly, and then realized that we had to propose a technical agenda and wanted to go after parts of it ourselves. But of course, there's also kind of the broader question of like, what do you do technically? This is why we are so committed to launching on day one with extremely strong privacy guarantees. Because you shouldn't trust me that if you hand all of your data to me, that I'm going to be a good steward of it. What you should instead know is that there's literally nothing I can do to use it in a nefarious way. That's a much more powerful guarantee. It's not this trust but verify thing. It's I can demonstrate to you that we have taken every measure humanly possible to prevent ourselves from training a larger model on your data. And so that every piece of data that we get from you is used at your benefit and we can't use it against you or use it to sell it to your boss. And I think that's different than a promise. We're trying to give an actual guarantee here so that we can't use the data in this way. That presents lots of novel challenges for our team. But I think it also presents some novel opportunities, both as to how we position ourselves and the kinds of things that we can do to help make your experience better as opposed to worse. We want these models to genuinely be aligned to you and loyal to you alone. And we're going to keep that vision centered as we continue to work on this. It is really one of the big kind of technical, perhaps even political questions of our time. We have AI models that are aligned to certain interests. There's a whole separate question of whether we can even align them to certain interests. And that, in my opinion, is an unsolved problem. But they happen to have certain goals, certain preferences. And those preferences are a kind of a mix of what the companies are interested in, what governments are interested in, and what end users are interested in. And the balance between which preferences should be strongest in the model, that is a very interesting question and something that, I think there's a lot of work to be done there. For example, I say in not that long, I expect us to have personal agents that can do our e-mail and our calendar for us. that agent, is that agent working on my behalf when I ask it to book a hotel for me? Or is there perhaps a kind of a corporate preference to book a certain hotel that OpenAI might have an agreement with, something like that? You could quite easily see the incentives of the model or the preferences of the model becoming modeled between what the end user wants and what the companies are interested in. Do you see a principled way to solve this? Or is this just like any other product where the company selling the product is interested in something and the consumer is interested in somewhat of the same thing, but the preference sets do not perfectly overlap?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.