Nathan Labenz: uncertainty

Transcript context

…Yes, so we are a 501 , so we have to keep our advocacy and stuff to a minimum. And I think a lot of Matt's strength is this impartial player. We're trying to be somewhat of a research university, tech accelerator kind of vibe. We don't want to play favors politically. That's not in anyone's interest. I think if people are doing that and they're trying to be the thing we are, they're doing a bad job. That said, I think currently, I believe David Kruger is going to be a mentor in the current program. And some of his research that he's going to be discussing is to do with, I guess, what sort of messaging and what sort of standards are actionable, right? But of course, I wouldn't say this is true advocacy. This is more masses supporting independent research, working with David Kruger, who has his new org, Evitable, not inevitable, Evitable, which is focused on some of these advocacy questions. I think Matt has to be pretty careful, in terms of obviously our 501c3 spending requirements for advocacy. We haven't spent anything on advocacy, for what it's worth. And also like, you know, ensuring this political neutrality. so that our fellows, our mentors, and all of our strategic partners just can feel assured that we are like, you know, we're solutions oriented, right? We are pushing for a particular outcome, right? And I think that AI safety being a political football is just a bad idea. And I applaud advocacy orgs, like in code and plenty of others, like, you know, perhaps CASE, et cetera, for their efforts to try and, you know, to, to, that's not, that's not mass as an organization. Yeah, gotcha. Toe in the water at most for now. Let's talk about the profiles. I both watched a talk of yours and read a blog post from about 18 months ago where you kind of sketch out the different archetypes of AI researcher that you have seen, and then also kind of map that onto the demands of organizations. And I don't know how much it's changed. in the last 18 months, if it all. But maybe give us the kind of baseline and then if there's any update, I'd love to hear how things are changing, especially, you know, I have in mind, of course, clawed code. And it may accelerate certain people. It may empower certain people to do things that they couldn't otherwise do. But yeah, tell us, first of all, how you organize your thinking about the kinds of people that you're bringing into the program. So, I mean, Matt's like I've talked about mentor selection committee. Well, we are fundamentally I think, this massive information processing interface. So we consult the very best people as much as we possibly can. And we try to build our own opinions, but we don't rely on them. We try and consult experts at every stage. So the paper or blog post you're mentioning, which was called Talent Needs of Technical AI Safety Teams, to construct that, we surveyed 31 different lab leads and hiring managers, whoever we could get, the most senior person we could get related to safety at every AI safety org we could find that was hiring. at that time. And we asked them, what do you need? And then we compiled all that survey, well, those interview notes into like three archetypes, right? This is just technical. We've since done this for governance. Expect that to drop soon. So those three archetypes were connectors, iterators, and amplifiers. So we chose the term connector because these people are bridging gaps between theoretical arguments for AI safety and theoretical techniques to make AI safe. and the empirical techniques to actually make it happen. So they're sort of like spawning new empirical paradigms to work on. Okay. No one is hiring these people. It's pretty rare because if you're good at that, then everyone knows your name and you're already hired. Perhaps you're already leading an organization and everyone wants to be an ideas guy, but very few people want to, to hire ideas guys. And these people typically, it's people like Bach Schlegeris, you know, AI control, uh, Paul Christiano, right, just a huge amount of, uh, resources he's produced and so on. You know these people, right? They typically have AI safety organizations they founded and lead. Then there's iterators, right? And this is not just engineering, right? Iterators are active researchers who have strong research tastes who are pushing the frontier, but they typically aren't creating novel paradigms based on theoretical models of things. They're typically advancing empirical AI safety. And you can even imagine iterators and technical governance agendas as well. So this is the majority of people that are working in AI safety today and also the majority of hiring needs in the future. And then there's amplifiers who I think like the closest example is like TPM archetypes. I'll say this for iterators like prominent examples include like Ethan Perez, Neil Nanda, Dan Hendricks. Actually, I think Dan Hendricks maybe crosses some boundaries there. But yeah, amplifiers, to distinguish them, they have more focus on amplifying people. And typically you'll find them on large research teams. And they're scaling the number of people that can be effectively managed and contribute to organizations. So a lot of maths research managers would fit this category, or TPMs at the various labs. And interestingly, they're actually quite in demand as well, particularly for labs in like the 10 to 30 FTE range.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.