Nathan Labenz: belief

Transcript context

…. I tried to make my product not do the heinous thing that the very best model developer is doing. Then everyone has no excuse, and they have to do that, and governments can compel them to, and so on. So I think making your model performance competitive enough that people want to pay the alignment tax, so to speak, seems like a viable strategy from that perspective. Now, of course, none of this is trying to justify the current frontier, the race of frontier models, which seems very reckless. Let's be clear. I think at the current pace of development that we're going to be in a lot of trouble. But this is one of those collective action problems. These companies have to coordinate to slow down. And there's international things at stake here as well, because now you have, you do have a US model versus China model developer kind of race now that they're in the running. So it's very complicated. And when you have these collective action problems, I think the main way you solve them is through governance. And sure, the lab leads could be probably even more collaborative. And definitely some of them are not advocating as strongly as they should be for slowing down and for having this kind of collective, you know, kind of sharing in the alignment benefits and and, you know, not pushing the frontier dangerously. But I do think this is ultimately a job for governments. Yeah. Quick. This might be a bit of a digression, but quick follow up on when you said companies are primarily doing this to make money. I actually would model them fairly differently than that. Take OpenAI, for example. Sam Altman has said, you know, we could burn $5 billion, $50 billion, $500 billion. I don't care. And we're making AGI. It's going to be expensive. I think that's almost a direct quote. And then when you look at their mission, I'm always struck by how the even just the way that they've chosen to define AGI to me strikes me as kind of inherently ideological. Like you could set your goal in any way, shape or form you might want to, and theirs is like explicitly in this frame of outcompeting human workers. And I think that they are sincere in their expectation that it's going to be good for everyone and it's going to free people from drudgery. And like, I certainly hope they're right about that. I'd love to live in a world where people don't have to do work they don't enjoy doing, but I don't know. I kind of I think of them more as like Obviously they're quite different too across the different companies, but I think of it less as like trying to get rich and more like trying to make a real mark on history is kind of the biggest summary that I would give for a lot of them. Does that resonate with you or not really? Maybe. I don't want to I can't really speculate on the psychologies of the leaders of these labs, let alone their share, like not shareholders so much as I guess like venture capitalist investors and everyone else they've made promises to, their clients and so on, their employees. I can't really speculate about that. I will say that given that the value of AGI is estimated as at least between 1 and 17 quadrillion dollars, I think that's a lot of money. It's a pretty big mark on history. I'm not sure if it even matters whether they're trying to make a big mark on history or make money. We can adopt Dennett's intentional stance about the AI companies. Be like, Okay, so what does it look like they're doing? If we were to conceptualize them as a coherent agent trying to do a thing, what is the thing that they would be trying to be doing? To me, it seems a lot like they're trying to make a bunch of money. But making your mark in history could also be valid. Though I would say, I guess in the world where I expect, I don't want to use any specific AI lab as an example, but I think in the world where an AI lab is trying very specifically to make their mark in history and not trying to make a bunch of money, I'd expect It might look identical, actually, to this world right now.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.