Nathan Lambert: belief

Transcript context

…I don't know if this is a relevant comparison for you, but OpenAI also recently released a weak to strong generalization paper where they actually talked about a few intermediate checkpoints for GPT-4. Any comments on the comparison between constitutional AI and weak to strong generalization? I didn't see the paper. I think I saw people criticizing it for like just being like safety washing from the fact that they're like talking about GPT-2 still, which is such a kind of like odd model to focus on. I didn't really look at the paper. I think that it's a thing with OpenAI. It's like they're sharing less than they know. So I think they probably have things that are pretty cool that they're doing internally. And I'll summarize for listeners who may not have seen the paper because it's impossible to keep up and everything. I do think that what constitutional AI and RLAIF represents is that we are starting to come to a point where it's just impossible for manual human preference data collection to scale. And the only way to scale this is to trust our AI overlords to model our human preferences. And constitutional AI was the first version of this. What the second version or what weak to strong is, is that anticipating a future of the need for super alignment, where the thing that we're trying to control is smarter than us. So you take GPT-2 and try to use GPT-4 to teach it to be smarter than itself, because this is what we're going to have to do in the future as well. When we are not, we're no longer fully in control. Are we the metaphorical GPT-2? No, we're like not even in the process anymore at the point of super intelligence. They're prepping. And they're saying this will happen. And humans will be like so far like in the dust that we just have no say in this debate.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.