Nathan Lambert: uncertainty

Transcript context

…I thought the communication around the Pareto optimal improvements was helpful in understanding that you do actually want it to be more helpful and honest while maintaining the same level of harmlessness or something. Yeah, right. Yeah, so that figure right at the top of the constitutional AI paper is worth seeing if you don't have it immediately pop into your head where they essentially compare like constitutional AI to other RLHF that they're doing internally. And that's something that most RLHF papers don't do is like they have little dots on the lines to indicate intermediate checkpoints and be really great to see more RLHF papers showing how per epoch or per half epoch of training because most RLHF is only a few epochs, at least in the open models, like what is happening there. People release checkpoints, but that's how we should be thinking about it because the optimizer is so strong and it's like we don't know what's happening in this kind of intermediate land. I don't know if this is a relevant comparison for you, but OpenAI also recently released a weak to strong generalization paper where they actually talked about a few intermediate checkpoints for GPT-4. Any comments on the comparison between constitutional AI and weak to strong generalization?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.