Ed Boyden: evaluation

Transcript context

…Let’s say you had $10 billion or $20 billion a year, and you would control your own agency, and you were starting all over again, but current institutions stay in place. What would you do with it? How would you structure your grants? You’re in charge. You’re the board. You do it. Yeah, three thoughts. The first thing that I thought a lot about — studying these past cases and then going through it myself — is thinking about peer review. What is peer review? When you propose a project, a bunch of your peers will then critique it. The problem that a lot of these daring-sounding projects encounter is that they sound bad during peer review because they’re so off the wall, or they bring together multiple fields that maybe nobody’s qualified to evaluate them. One thought is, what if — instead of taking people’s opinions and then just sort of combining those opinions, and then, okay, you’re in or you’re out in terms of getting the money — what if we take a step back, and we think about why the peers are thinking this way? If somebody critiques a proposal, but they’re doing it from a vantage point that doesn’t see a certain part of the proposal as valuable because they’re missing an underlying piece of knowledge, or they’re evaluating a proposal — based upon opinion — that, if we think about the logical underpinnings of it, the rationale is actually pretty solid in terms of its being linked to ground-truthable sciences, like physics and chemistry. In other words, if we take a step back and apply more logical principles of evaluation to the outcomes of peer review, can we actually improve the ranking of these proposals? This is something I’m thinking a lot about right now. As I evaluate people and evaluate ideas that people propose to me as well, I’m trying to hone those skills in myself. That’s one of the three things I would do. How much more productive could science be on a per scientist basis? Let’s say you could redesign everything: refereeing, universities, how grants are made. Can we double the progress of science? Just boost it by 10 percent? It seems science is slowing down in many ways.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.