Nathan Labenz: belief

Transcript context

…Yeah, I mean, some of the oversight control researchers might be more known because this is one of the things that a lot of the companies, big companies are pursuing. So we have people like Buck Schlegeris at Redwood Research and his whole team are part of that. Ethan Perez and Sam Marks and many other people at Anthropic. We have Eric Jenner and David Linder and Victoria Crokofeno and many at DeepMind. So yeah, just tons of people doing oversight control research. For interpretability, obviously we have Neil Nanda, We have some of the Timaeus folks like Jesse Hugo and Dan Murphitt. We have, of course, the Goodfire people like Lee Sharkey, longtime mentors. Some of the people from Simplex who are doing some of the interesting, very interesting stuff, like Adam Shea and Paul Rikers. And they're doing like, yeah, it's all of Timaeus. I would say that this is some of the more interesting, kind of maybe more moonshotty, but very promising interp research bets that I have on the side as well. For evals, we have people from Meter. We have people from UKAC with people from Apollo Research, Marius Hoban, plenty of others there. Yeah, I could go on. There's, there's some amazing researchers there. We also have some harder to categorize research. Yoshua Bengio's whole team at Law Zero, plenty of Yoshua Bengio himself and plenty of others are there. And we have some AI sentience research as well, digital minds. So Patrick Butlin at Elios AI, Kyle Fish and Anthropic. Yeah, it's a very exciting program. Yeah. That's quite a who's who in a couple past podcast guests and a couple that I took note of as maybe need to put down an invitation. It seemed like the balance, if I was kind of categorizing those right, it seemed like a majority would be in that first empirical category. Is that, do you think it stays that way? Or, you know, your comment on kind of ultimately this is a job for governance tracks some of the Honestly, the more MIRI line these days. I think the MIRI line today would be like, we don't really have time for that much research. We need to just go straight for the global treaty. You're not obviously quite so confident in that direction, but it sounds like you do believe ultimately that there is a major role for governments to play, and you're starting to move more in this governance and policy direction. Do you see that as-- is that going to be the biggest growth area reflecting that worldview, or how do you expect the balance of these different areas to evolve over time? I actually can't necessarily say. Well, okay, I can speculate, but I'll say this. We have had actually about the same proportion of governance researchers, give or take a few percent, for the last two years. It hasn't changed by fraction. So we are quite on track for, you know, continuing the same trend potentially. Part of the reason is like we are based in some of the, particularly in Berkeley, SF Bay Area, this is a big technical hub. There are other programs that have had more, you know, a deep governance focus. Like we have gov AI, their classic fellowship. We have IAPS, we have of course RAND CAST, this large program run out of RAND, and plenty more besides, but Yeah, and of course, Horizon Fellowship for US Policy Careers. And these have also kind of existed for longer than we were around. So at a time when we were basically the biggest fish in town, which we are still in many ways in terms of funding and I think in terms of prestige as well. For technical AI safety, we're the biggest and best program. But I would say that for governance, there's always been a bigger fish. And so we've never felt that it's necessary to overweight governance beyond what our mentor selection committee indicates. In fact, that's the primary determination of what tracks get selected, right? Is our mentor selection committee, which is somewhere between 20 to 40 top researchers, strategists, et cetera, org leaders that we survey. When everybody applies as a mentor, we decide the mentor level based on the feedback from our mentor selection committee, who gets in. With some additional caveat that like we also have some diversity picks. and minimum requirements because we want to support a great breadth of research and we think that the mentor selection committee on the whole might be biased in some ways as well. So we try and like really talk to the experts when it comes to picking the agendas. And it so happens that like governance researchers have historically been like relatively low rated by our committee, which contains many governance researchers. I think I would go so far as to say that governance research is harder to do well. in some critical sense. It's harder to see what is the actionable thing to do in some ways. Now, everyone who has their specific governance agenda, I would say, doesn't feel this way for a good reason, right? Within their agenda, they have clear, actionable things to work on. But I think on the whole, there's just so many more possible technical directions to pursue that are high leverage in some ways as well. I think a lot of the governance stuff is like, oh, we're trying to build this is, this is not talking about advocacy now, right? This is talking about technical governance. We're trying to build technical governance solutions such that if an administration so deems them worth, you know, uh, uh, deploying that we have the capacity to do that. We actually have the solutions that can be deployed, which is very important, right?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.