Nathan Labenz: belief

Transcript context

…I mean, I think this is a huge topic. One starting point would be like you're kind of gesturing out. I think a lot of the reasoning in this area is it's pretty shallow. It's just, well, like company has the good vibes. So I'm going to really encourage you to try and be more objective than that. Like think all these companies like political actors, they have all the leaders have many incentives. They have many different goals, not all of which are noble. And then try and just like actually get down to the level of specific decisions and track record that they have. And did they actually make the hard decision in that case or not? and try and. be grounded in objective decisions as much as possible. Or when they made this commitment, did they follow through on it? What do people say about this person's character who know them well? And actually try and take a view on it rather than just go with vibes. That's about assessing the company. With assessing yourself, that's even harder because you're going to be super biased there. This is just one small technique, but I do think cultivating some close friends who are willing to call you out on stuff is really helpful for this. And most people won't really call you out. They'll just go along and they want to have a friendly relationship. So if there are any people in your life who will do that, I think that's really valuable. I mean, potentially making commitments to them as well about almost like a written down commitment. If this type of thing happens, I will do this. And that can make it. I think it's very easy for kind of your standards to slide over time gradually, right? Ways to just have some kind of pre-commitment in the line where you're going to reassess, I think is really helpful as well. I guess a fourth thing that comes to mind is like you will become more similar to the people you work with. So again, that just means really taking seriously the context. And like we said with the 10 people in the inside, most people can't, if they think there's like a company that's being really reckless, most people can't actually hack like going in there and trying to make it better. That's like actually a very difficult thing to do. So I think be skeptical of your abilities in the prevailing culture, in the people around you. And so just focus on picking the right culture in the 1st place. One other argument that people have made a lot who I think are especially skeptical of AI companies, but I guess just maybe just have a hype doom as you framed it earlier, is that safety work is ultimately still counterproductive. Sometimes that's framed as saying safety ultimately is just an accelerant of the capability frontier. For example, we had things like RLHF that were supposed to be a safety measure. And in some sense, they do make the model, any given individual model, safer to use. But then they also have had the property of making the AIs much more useful and just accelerating the overall pace of development in the space, bringing lots more resources in as it became clear that, hey, these things can in fact be harnessed and become super useful. And I wonder how you feel about that now. The argument that those people usually make is just try to shut the whole thing down. And this is not my point of view to be clear, because I do really value the upside that we're getting. But There is some truth to that argument. I can't quite dismiss it. How would you assess that argument and how moved people should be by the sort of even AI safety, even technical AI safety work is still bad argument. I mean, technical AI safety definitely does seem to sometimes improve capabilities and therefore accelerate AI overall. The question is just, are you getting enough benefit out of it to make up for that cost? Because world where we don't do any safety research also seems pretty scary, right? My sense is it's best to have a thriving safety ecosystem who are really tracking the biggest risks and trying to do something about them, even if they will also accelerate things a bit even more compared to a counterfactual where basically things are still going very fast, but we don't have a safety research ecosystem. It does depend on whether you think, if you just think alignment or AI control is ultimately impossible, then yeah, that doesn't make sense. I think a bigger point I might make about this is like, I guess I come at this from a perspective of just we're very uncertain about all of these different priorities and what's going to happen. And that, at least for me, makes me intuitively want to see a bit of a portfolio of efforts across different scenarios. And so I'm like, I'm pretty into the idea of there being some people who are really trying to build support for some type of international pause. I think like a one year pause at the point at which an algorithmic feedback loop becomes possible or some kind of temporary pause seems like it could be extremely helpful. At the very minimum, that's like a thing that I think many people could agree upon. But then I do also want there to be a lot of effort going into the scenarios where, no, this is happening, like it's not really pausing. and just betting 100% on the pause seems that's leaving a lot of like low hanging fruit on the other scenarios on the table.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.