Jeff Beck: preference

Transcript context

…On that, there's an AI literacy thing because AI has moved so quickly now that certainly my parents don't understand anything about it. But by the same token, policymakers don't understand anything about it. And there are people saying AI is going to kill everyone. And there's people making negative arguments. There's people making positive arguments. There's a bit of a fog of war now because there are so many people saying different things about AI. How should they make sense of all of this? We are now well outside my area of expertise, so I'm just gonna say that before I say anything else. AI is developing very quickly, But I am much more concerned about what people will do with the new technology than I am with what the technology will do all by itself. I don't have this big concern about, I don't really believe that like Skynet's gonna take over, the internet's gonna suddenly become conscious and kill us all. Right? In part because AI is not that advanced, but also because we are telling, we are still in the position where we specify the goals of the system. And that will likely continue for a very long time. And it will always be the case that these systems, you know, will can be, you know, are are subject to review. We will always keep an eye on them. They will always at least initially be be be released in relatively restricted domains and where we're where we're test where where we're keeping a a close eye on what it is that they are and are not doing. So I don't worry too much about, like, the going rogue. I worry a lot more about somebody building you know, it's sort of like a virus, which we already have to deal with. Like, somebody builds, like, some insane virus and, like, takes down the Internet. I'm more worried about malicious human actors than I am malicious AI actors because at the end of the day, all of these algorithms, they simply do what they are told. Right? We train them. We tell them here's your objective function. As long as we are specifying the objective function and we understand the objective function, we're probably going to be okay. I think the safest way to deal with AI concerns is to tell people, hey, look, this AI is just doing what we told it to. We set it up to make really good predictions and to achieve these outcomes. Now is it dangerous to specify these outcomes without being very, very, very careful? Yes, it is. This is the whole, hey, Skynet, end world hunger, and it kills all humans. Is a real possibility, but whose fault was that? The fault was the person who, like, was very, very naively specified their goals. There are, in fact, relatively straightforward ways to specify the the reward function that that don't run that risk nearly as badly. And the best 1 is so are you familiar with maximum entropy inverse reinforcement learning? I like to call it active inference because it's really similar. And so there, what you're doing is you're basically observing someone's policy, and then you're trying to do a maximum entropy model, you're doing maximum entropy model on the reward function itself. At the end of the day, what ends up happening when you do this is, this is why it's like basically just like active inference. You get a reward function, so you have some, you know, organism or whatever and you're trying to do this for it, it's got some stationary distribution over actions and outcomes, right? It's inputs and outputs of the stationary distribution. ome, you know, organism or whatever and you're trying to do this for it, it's got some stationary distribution over actions and outcomes, right? It's inputs and outputs of the stationary distribution. That becomes your reward function. Like not directly, there's some math involved, but basically your reward function is a function of the steady state distributions over actions and outcomes. So we could do this, right? We could take the current manner in which humans are making decisions, and we could write down, right, what's the stationary, what is the current estimate of the stationary distribution over actions and outcomes? So this would include things like everyone's getting, this number of people are going hungry, this, you know, and and, you know, all of the stats that describe like the inputs and outputs to our policy make, you know, to our policy. Then we could just ask an AI, your reward function is the 1 that results in the same outcome that we currently have. Right? On average. And it would execute it and it would and and to the extent that it works. Right? It it it would it would ultimately result in a in an AI algorithm that just sort of is like mimicking human behavior, right, or at least achieving the same outcome that we were achieving before. Now here's the safe way to like improve the situation. You don't say end world hunger, right, you perturb that distribution over outcomes, right, and just just over outcomes a little bit, and then you evaluate the consequences, right. It's it's all you're doing. You make these little changes in the reward, in an empirically estimated reward function, right, rather than just sort of specifying 1 by hand because that's the dangerous thing.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.