Tim Scarfe: evaluation

Transcript context

…You know, that that all sounds great. It's just not the language that those of us who do research would use. I mean, we would think in those terms a little bit, of course, but we would try to turn it into some kind of an equilibrium or optimization problem and here's the information that's available and here's the data and here's the power and the, you know, the the error rates. And we try to put a little bit of structure around it of that form. And and, you know, there's always this creative moment. Like, remember when in high jumping, I used to be high jump interested in high jumping when I was a kid. And, you know, you you would go up to the bar and you'd jump over it in various ways and there were the the barrel roll rolling across, you know, was the technique the Olympians were using. And then there's this guy, Dick Fosbury came along and he says, no, if I go backwards, I can do better. And no 1 had thought about doing that. As soon as he did it, everybody did it and that's and and, you know, it went up like by half a meter or something. I don't know. And so what process led to that? Was it an understanding process? It was just a little bit of let's try something different mixed in with the ability to try it out and to do tests. So a huge amount of industrial planning is try it out and see what works. Those are called AB tests. And those are done all of the time, and I've got nothing against that. It's not based on understanding, but it's led to optimized systems that can do things that, you know, people hadn't thought about before. So a blend of that with understanding. But just understanding, you know, I'm I I was a cognitive scientist and I, you know, I'm interested in neuroscience. I'm 1 should be interested in those things. They're fascinating. They aren't they aren't the leading edge of thinking how to build systems that, you know, work in the world. And they're not the leading edge of trying to believe in the next generation systems. If you put a lot of people keep saying, well, we got to put logic back in or symbols because that came from the that came from our previous kind of view of what humans are doing. And probably humans are capable of doing some logical reasoning and probably have some symbols, whether they're kind of built in some complicated network or they're reified somehow, I don't know. My intuition is as good as yours. But really, the goal is to I tend to be an engineer at heart, a mathematically inclined engineer. I wanna say, what are you trying to achieve? Are you trying to displace teachers? Are you trying to make doctors better? What are you trying to do? And what would be the abstractions and the points of entry into that problem? And then how can you pull back from that and do it in some general way that's elegant and will inspire others? It's so interesting seeing different scientists, you know, from a multidisciplinary perspective attack this problem. Physicists, for example, they they work very, low level and they talk about, you know, the the the dynamics of particle systems and and whatnot. And what I'm really fascinated in, I mean, you come at it from an economics perspective, which is traditionally dominated by this agential lens, and you talk about equilibria and incentives and and so on. How how does that come into it? How how would you take a very complex system and almost kind of decompose it into this new frame of thinking? It's not been done really enough for me to have, you know, tons and tons of of great examples. But, you know, we've been looking at kind of modestly scaled examples where there's a so for example, we looked at a little bit of drug discovery and kind of the regulation. I'm a pharmaceutical company, I test out all kinds of proteins and I throw them in animals and maybe a few humans to sort of see what's working and I have some understanding, quote unquote, in other words, I know somebody of the evolutionary biology behind it and so on. That guides me. But at some point, someone's got to really test this out in the real world and decide regulatory agency's got to come in and say, yeah, that goes to market or it doesn't. So now you've a kind of tangled web of scientists and pharmaceutical companies, not just 1 but many, many of them, and proteins. And now you've got to think about how that system is behaving. Hopefully the regulatory agency is is trying to, overall over the entire system, have the number of false positives be low and false negatives be low. That's what the goal is of the problem. So it's a statistical problem. Oh, but wait, a classical statistical problem, you would just go gather IID independent, identically distributed data from some source. Here, no, the data is coming from the self interested pharmaceutical companies. What's their motivation? Money and whatever. Maybe the hell they want to help people and money. And all that's kind of hidden from you as a regulatory agency. So now the economic mindset kind of comes into play. He says, Well, it's hidden from me but it's not arbitrary. I can kind of probe in various ways. And so that becomes very economic. So, you know, economic economists think about how you set prices. You know? So if I got a lot of people coming on my airline, there's 1000 people that just arrived who wanna go from here to London. Every 1 of them has a different price point and that price point will shift in the moment. How how eager are they to get? It's not just because they have a lot of money. It's because they have needs. And I don't know what those are. So what I do is I set up various services and various prices that kind of bracket the possibilities so that overall, it's likely I'll make enough money and they will everybody will be kind of happy and the service will go forward in life. That's what so it's a blend of knowing a few things and admitting that you don't know other things, but putting it in a system that actually can work with that kind of mix of asymmetries and and incentives. So the incentives there are that there's a certain service and price. If you pick that 1, you're likely to be able to get on the airplane, you're likely to have the goodies you need or whatever. And that then doesn't make you do something, it incentivizes you.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.