Cristopher Moore: recommendation

Transcript context

…Right. Yeah. So I I I have thoughts about this, and maybe this is another conversation. I don't think these things should be inscrutable. I mean, I I I think that there is a range of applications. Right? If you recommend movies to me using a black box and I like the movie, everybody's happy, that doesn't bother me. Right? Maybe if I were a filmmaker, I would want to know more, but it doesn't really bother me as a consumer. At the other extreme, if you are putting me in jail even though I've not yet been found guilty of a crime, or if you're using AI to help find me guilty of a crime, you know, we have these things in the Bill of Rights that say I I should be able to confront my accuser. Witnesses. I should be able to contest evidence. And the interesting thing about these things, this is what people call procedural fairness. And the interesting thing about the criminal justice system is we explicitly care about things other than accuracy. Right? So for instance, we've all watched TV shows where the guy actually did the deed, but the police planted the evidence. They violated the rules of evidence. They knew he was guilty. They wanted to put him away, and then they crossed the line. And because of that, he got to walk. And in our society, we think that's how it ought to work. Right? Because we don't just want to be accurate in putting away guilty people and releasing innocent people. We want to have a certain relationship between government and its citizens. We want to have rules about how can the government, surveil you, investigate you. And that's really profound, right? And that's just, you know, how do you optimize for that? How do you even mathematize that? In some of the work on fair machine learning, you know, people look at statistical notions of fairness. Oh, we have this group of people. We have that group of people. I'm a little bit disturbed by the assumption that everybody belongs cleanly to 1 of these 2 groups. I think that's part of the problem. But to the extent that we can divide the world into subpopulations, and it's like, well, we want the false positive rate to be equal or whatever. Well, that's a constraint. We can add that to the model. We can tack that onto the algorithm, and people have done lots of good work in that direction. But I'm really fascinated by these other harder to mathematize notions, not just of fairness, but, you know, I mean, what do we really want these systems to do? ne lots of good work in that direction. But I'm really fascinated by these other harder to mathematize notions, not just of fairness, but, you know, I mean, what do we really want these systems to do? 1 interesting fact, which I recently learned from a guy named Mark Knaeus, who has a PhD in aerospace engineering and then went to law school and became a public defender, I mean, is that a lot of the software products which were being used to do DNA testing, specifically this thing called probabilistic genotyping where it's been a couple of days, there are multiple people who pass through the scene, the DNA has fallen apart into pieces, you know, how do you then there's there are some choices to be made here about how this is not a perfectly clean math problem about was the defendant at the scene of the crime in that setting. Many of the software tools that are used for this, there are kind of 2 or 3 popular ones, Some of them have never been or at least until recently were not independently tested by anyone. Many of them were not open source. They were proprietary products, and they sometimes disagreed with each other. Right? So what is the right metaphor here? Is this, are these things expert witnesses that you can cross examine? Not really. Are their designers the witnesses? Do you cross examine their coders or the bioinformatics behind them? You know, if they disagree with each other, how are judges and juries supposed to evaluate which 1 is better? So for me, I I like the idea of transparency, which for me is a stronger word than explainability or interpretability. I agree transparency is a moving target. In some settings, it might just be, has some independent agency, consumer reports, or underwriters laboratories tested this thing, and can they verify the vendor's claims that it works? In some in some settings, that might be enough. And in a lot of settings, even that is missing, right, from things that are being used right now to make important decisions about people. In some other settings, I really want to be able to look under the hood. And, yes, I know deep networks are hard to interpret, and but at least it's a start. If I can look under the hood, I can do these sort of fMRI experiments like, you know, like this Othello paper where people try to do the tomography and figure out what kind of model it's building. I think that's a very interesting line of work. So I think that as humans, it would be very good for all of us, especially if we want a democratic society where we're making kind of informed collective decisions about when to use these things, if to use these things, and what settings to use them. We should all try to understand these things as well as possible. There are multiple sources of gaps in our understanding. Some gaps are there for honestly good reasons, like deep networks are hard to understand. Some gaps are there because of intellectual property and because people don't want to reveal how these things work because they want them to be proprietary. I am not very sympathetic to that second kind of gap, and I think that kind of gap should be closed. because people don't want to reveal how these things work because they want them to be proprietary. I am not very sympathetic to that second kind of gap, and I think that kind of gap should be closed. I don't think we should be using opaque proprietary tools to make decisions that affect people's fundamental human rights. I think it's a continuum. Like in health, it's interesting. Like if you are using a proprietary tool to diagnose my cancer, well, I mean, I'm a geeky guy. I'm really curious how it works. I would, you know, if it's been independently tested by people who are not paid by the vendor of the system and it's really led to good outcomes, even if it's a black box, I might go along with it because I wanna live, you know? Yeah, I don't know. I mean, think it's a continuum, like what level of transparency we would demand, but when we get into sort of constitutional rights, I think we should demand every possible form of transparency.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.