Michael I. Jordan: evaluation

Transcript context

…Well, suppose another thing that doesn't help is that these these systems are like soup, and there's even a field called mechanistic interpretability that tries to kind of dig into the soup and and it's almost like they're they're searching for UFOs. They're trying to find these principled circuits that do reasoning or or do whatever the thing is. And I guess you could say cynically that it's not like when engineers build a bridge. Well, I'm I'm a little less negative than that. I I don't think it's bad to build systems you don't understand. But then you've got to kind of put things around it. And the things that are beeping around are like buzzwords, like AI safety. It's a buzzword. Okay? What you really need I mean, a human, you can't explain to me why you picked this Airbnb over another 1 or whatever. All the choices you've made today are inexplicable to me. They come out of your brain. And I don't need to know all the whys and wherefores of your choices. And what I need to know is that you're somewhat predictable and that if I make certain options available to you, you're likely to take this 1 versus this 1 and therefore I can make my own plans and we can start to interact and so on. So that's part of economics is the economic style of thinking says, I don't understand all these other entities out there but there are certain rules of thumb that I can use or quantitative predictions I can put in place that allow me to interact and not get hurt and even get value out of it. So no, don't think it's necessary to understand all the details. Now, the inputoutput behavior you often have to understand better than we can now. For example, if I'm denied a loan at a bank and the bank uses this big AI program based on past data, I want to know why. And why doesn't mean that you look in the internals and show me some circuit. No one's going to want that. They're going to want, well, here's 50 people that are pretty much like you according to the embedding we're using in this big network. And of those 50 people that are like you, some of them got the loans, some of them didn't. And here, let me just show you what those people are like. And start to say, Oh, I see. They differ from me in this way. That's actionable to me. I could now change things. So you have to build systems around this predictive system. That's a nearest neighbor system, for example. And that system will supply what people might consider more like an explanation. And so it's not just trying to go in the internals or something. Again, engineering is there's certainly thermodynamics and lots of things are understood, but lots of phenomena were not understood for a long, long time. You mix up a bunch of stuff and certain waves are created and certain things happen exploit that and move on. But you understand something about input behavior constraints and so on. I think the current generation of neural nets will continue to they have very nice scaling behavior, they'll continue to be there. But they really have to be thought of as a part of a bigger ecosystem. And then you kind of ask, well, what can the neural net do in this context and what's it missing and what if I have multiple of them and How do they engage with each other and with us? What transparency is needed for the overall interaction to be an effective 1, whether or I understand all the details or not? For some reasonand correct me if I'm wrongI have an intuition that behaviorism is bad, that just by not having any mechanistic understanding and only looking at the outputs. There's the famous example, isn't there, of of of the hen didn't know his neck was gonna be broken. And 1 example of this actually is AlphaFold. So I I interviewed John Jumper last week at Google.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.