Andrew Gordon Wilson: belief

Transcript context

…I was gonna ask you why you were theory because I know you're a theory guy. Are you a theory guy or or an engineer? Presumably, you're both. But, you know, deep in your bones, are are you an engineer or are you a scientist? It's really hard to choose. In some sense, both. I'm mostly driven by trying to understand things. And so this can be done in a variety of ways. A lot of our papers empirically try to understand model behavior. And so I feel like this is a scientific approach to to machine learning. And 1 thing that really motivates me about this type of approach is whatever you learn will never go obsolete. So quite often, newcomers to the field and even, you know, very experienced researchers feel distressed at the rapid pace in the discipline where you see methods getting published at a conference and then becoming obsolete within a month. Or when you go to the conference, everything you're seeing is sort of, you know, been replaced by some other algorithm. And wonder, okay, well, is there any point to me sort of investing myself significantly in building a model if I know that it's not gonna be used by anyone for any long period of time? And I think 1 way to address that is really to try to combine what you're doing with an understanding of why things are working. So if you're building a model that gets better performance on some problem, there's a reason for that. And if you can understand the reason for that, that understanding will outlive that specific model and how widely it might be used. And I'm really hoping to do research that will be relevant in hundreds of years from now. And so I think these questions around model selection for example and Occam's razor, people will never stop asking. Hopefully, they'll be able to go back and read not just my work, but like work that's been done in this space and think, okay, this is useful to me in thinking about how to approach some of these questions. And so, in this respect, I would say I'm a scientist and I try to combine classical theory with empiricism towards understanding model behavior. And I think if you really understand something, hopefully it's something that you can demonstrate in practice. And so I also try to combine some sort of practical demonstration with a lot of the work that I do. And this process also involves engineering. And sometimes understanding those low level engineering details becomes really fascinating and leads to unexpected intuitions about the principles behind model construction. I think this is something that's perhaps underappreciated. Like, quite often, you can have a great idea and whether it works or not depends very significantly on all the low level details like numerical stability and other things like that. And when you get really deep into those details, sometimes you can discover things at a higher level that are also very significant and how we should think about model construction and algorithm design. Very cool. Maybe well, I was just gonna share with you. There's a a duality that Shannon pointed out that that you they may like and, you know, it's a duality between past, future, knowledge, control. He said he said, we have no knowledge of the future, but we can control it. We have knowledge of the past, but we cannot control it. And and I took that and related it to science and engineering. So the way I look at the 2 sides of that coin is scientists leverage control to gain knowledge. Engineers leverage knowledge to gain control. So you can be both because the the new knowledge that you collect allows you to better control the environment, the future of the world, and collect even more learning. Right?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.