Tim Scarfe: evaluation

Transcript context

…I mean, just not a good way to think about engineering. I mean, if you were a chemical engineer back in the forties and fifties, saying we're just gonna throw a lot of stuff together and make it work. Well, you could do it, but you'd get a lot of explosions and a lot of economically nonviable things, you'd hurt a lot of people. And I think a lot of these people are not thinking about all the people that are being hurt already of Facebook and so on. It's damaged a lot of young people. A lot of teenagers are having mental health problems. And this is just not something that isn't talked about by computer scientists at all. And now we're talking about yet another level of displacement of jobs may go away, but that's tough. It'll create new ones, of course, like always. You know, I just don't like to talk that way. And, you know, so you gotta say, well, step back a moment. What is your point? Are you trying to create a new kind of market where people could come in and have their talents valued and appreciated and or bids could be put out for things that people might need, and collaborations could emerge, and there could be producer consumer relationships being explored and understood and developed, and this could all be a mix of computation and humans. I think eventually we'll all kind of merge, but along the way, just doing something so disruptive with all of these metaphors that's not good social science or not good mathematics, It's just metaphors. And yes, you can build it because the previous generation of people created these amazing things that collect data. And we can do gradient descent on it and ad hoc architectures. And yes, that works. It's amazing. But let's not give so much credit to the people that did that. It's the people 20, 30 years ago who did that. So the current generation is just way too know, there's not much thought going on, not much intellectual stuff. It's just yeah. It's possible to build it. It's possible to steal the data from wherever you want to because that's what the Internet allowed to happen and not return any value to the person who originated the data. It's possible to run greedy descent on that but you need huge amounts of money but it's now possible to get it from people who aren't thinking very deeply. And so maybe seeming more dark than I want to. I mean, there's a lot of good builders. But we also, every previous era of engineering development, electrical engineering, chemical, mechanical and all, had some builders, but they had a lot of concepts and they had a lot of thinkers. In fact, all of those engineering disciplines had something like Maxwell's equations or Newton's equations to help them kinda here, no. It's just people that are very smart and who can code, and then have lots of intuitions, and and it seems to and I don't ever see anything that feels deeply intellectual to me. It feels like science fiction. Well, suppose another thing that doesn't help is that these these systems are like soup, and there's even a field called mechanistic interpretability that tries to kind of dig into the soup and and it's almost like they're they're searching for UFOs. They're trying to find these principled circuits that do reasoning or or do whatever the thing is. And I guess you could say cynically that it's not like when engineers build a bridge. Well, I'm I'm a little less negative than that. I I don't think it's bad to build systems you don't understand. But then you've got to kind of put things around it. And the things that are beeping around are like buzzwords, like AI safety. It's a buzzword. Okay? What you really need I mean, a human, you can't explain to me why you picked this Airbnb over another 1 or whatever. All the choices you've made today are inexplicable to me. They come out of your brain. And I don't need to know all the whys and wherefores of your choices. And what I need to know is that you're somewhat predictable and that if I make certain options available to you, you're likely to take this 1 versus this 1 and therefore I can make my own plans and we can start to interact and so on. So that's part of economics is the economic style of thinking says, I don't understand all these other entities out there but there are certain rules of thumb that I can use or quantitative predictions I can put in place that allow me to interact and not get hurt and even get value out of it. So no, don't think it's necessary to understand all the details. Now, the inputoutput behavior you often have to understand better than we can now. For example, if I'm denied a loan at a bank and the bank uses this big AI program based on past data, I want to know why. And why doesn't mean that you look in the internals and show me some circuit. No one's going to want that. They're going to want, well, here's 50 people that are pretty much like you according to the embedding we're using in this big network. And of those 50 people that are like you, some of them got the loans, some of them didn't. And here, let me just show you what those people are like. And start to say, Oh, I see. They differ from me in this way. That's actionable to me. I could now change things. So you have to build systems around this predictive system. That's a nearest neighbor system, for example. And that system will supply what people might consider more like an explanation. And so it's not just trying to go in the internals or something. Again, engineering is there's certainly thermodynamics and lots of things are understood, but lots of phenomena were not understood for a long, long time. You mix up a bunch of stuff and certain waves are created and certain things happen exploit that and move on. But you understand something about input behavior constraints and so on. I think the current generation of neural nets will continue to they have very nice scaling behavior, they'll continue to be there. But they really have to be thought of as a part of a bigger ecosystem. And then you kind of ask, well, what can the neural net do in this context and what's it missing and what if I have multiple of them and How do they engage with each other and with us? What transparency is needed for the overall interaction to be an effective 1, whether or I understand all the details or not?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.