Ryan Kidd: belief

Transcript context

…Yeah. That's quite a who's who in a couple past podcast guests and a couple that I took note of as maybe need to put down an invitation. It seemed like the balance, if I was kind of categorizing those right, it seemed like a majority would be in that first empirical category. Is that, do you think it stays that way? Or, you know, your comment on kind of ultimately this is a job for governance tracks some of the Honestly, the more MIRI line these days. I think the MIRI line today would be like, we don't really have time for that much research. We need to just go straight for the global treaty. You're not obviously quite so confident in that direction, but it sounds like you do believe ultimately that there is a major role for governments to play, and you're starting to move more in this governance and policy direction. Do you see that as-- is that going to be the biggest growth area reflecting that worldview, or how do you expect the balance of these different areas to evolve over time? I actually can't necessarily say. Well, okay, I can speculate, but I'll say this. We have had actually about the same proportion of governance researchers, give or take a few percent, for the last two years. It hasn't changed by fraction. So we are quite on track for, you know, continuing the same trend potentially. Part of the reason is like we are based in some of the, particularly in Berkeley, SF Bay Area, this is a big technical hub. There are other programs that have had more, you know, a deep governance focus. Like we have gov AI, their classic fellowship. We have IAPS, we have of course RAND CAST, this large program run out of RAND, and plenty more besides, but Yeah, and of course, Horizon Fellowship for US Policy Careers. And these have also kind of existed for longer than we were around. So at a time when we were basically the biggest fish in town, which we are still in many ways in terms of funding and I think in terms of prestige as well. For technical AI safety, we're the biggest and best program. But I would say that for governance, there's always been a bigger fish. And so we've never felt that it's necessary to overweight governance beyond what our mentor selection committee indicates. In fact, that's the primary determination of what tracks get selected, right? Is our mentor selection committee, which is somewhere between 20 to 40 top researchers, strategists, et cetera, org leaders that we survey. When everybody applies as a mentor, we decide the mentor level based on the feedback from our mentor selection committee, who gets in. With some additional caveat that like we also have some diversity picks. and minimum requirements because we want to support a great breadth of research and we think that the mentor selection committee on the whole might be biased in some ways as well. So we try and like really talk to the experts when it comes to picking the agendas. And it so happens that like governance researchers have historically been like relatively low rated by our committee, which contains many governance researchers. I think I would go so far as to say that governance research is harder to do well. in some critical sense. It's harder to see what is the actionable thing to do in some ways. Now, everyone who has their specific governance agenda, I would say, doesn't feel this way for a good reason, right? Within their agenda, they have clear, actionable things to work on. But I think on the whole, there's just so many more possible technical directions to pursue that are high leverage in some ways as well. I think a lot of the governance stuff is like, oh, we're trying to build this is, this is not talking about advocacy now, right? This is talking about technical governance. We're trying to build technical governance solutions such that if an administration so deems them worth, you know, uh, uh, deploying that we have the capacity to do that. We actually have the solutions that can be deployed, which is very important, right? ch that if an administration so deems them worth, you know, uh, uh, deploying that we have the capacity to do that. We actually have the solutions that can be deployed, which is very important, right? But I would say that like, don't rule technical research out because it is Like, especially if we have something like a regulatory market or even like warning shots that cause the public to wake up and tell Congress to regulate this stuff, right? We have to have technical solutions ready to deploy to make these systems safer. And the cheaper we make it, right, to make systems that extra degree safer, right, to lower the alignment tax that companies have to pay to train and deploy their systems to be safer, the more likely they are to do it. when they come under pressure, either internally, externally, or whatever. So I think that lowering the alignment tax via technical research is super important still. Also, if this alignment MVP plan is going to work, we have to have a bunch of directions for things to be iterated on by these AI assistants, or humans calling teams of AI assistants, as it's more likely to be. you actually have this massive interplay between technical research and governance research, where things like evals and safety cases built on technical AI safety solutions are things that can actually be tangibly put forward in policy proposals, right? And can convince policymakers of demos and evals and model organism honeypot traps, right? Where AI systems deceive the users or whatever. This is what convinces policymakers to make policy and gives them a tangible target for their policy to work on, right? So there's a clear flywheel here. So I would say like, do not roll out technical research. And there is a reason why MATS has so many more technical mentors. And that's just because it seems like on the whole, our mentor selection committee thinks that, you know, I guess on average, a technical portfolio is worth pursuing.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.