Speaker unverified: observation

Transcript context

…Do you have an intuition on which areas work better for which? For example, you mentioned the physicians are expert. What about maybe like code or, yeah, you also have a multi-model working on, so like image generation is like, or does this apply to any modality, any subject? That's an open research question. The intuition in general is like, for instance, for code, because this is factual, you can check if the code is correct or not, RLHF is not the way to go. You prefer to do like supervised fine tuning as a human to write the code. But in fact, because humans make mistakes, because actually even in code, there are some preferences that emerge like that. And maybe for some other reasons that we don't know, RLHF is so much more scalable. It costs less, it's easier, that it leads in general to just better performance. And maybe we can come with a compromise. We actually suggested teacher forcing in Llama 3, a new method that kind of fills the gap between, not teacher forcing, sorry, teacher critic. Teacher forcing is a good way to train the models. Teacher critic where it reconciliates and unifies supervised fine tuning and RLHF, so that when you do human preference, and you have two outputs, but both are very bad in the code, for instance, you will ask the human to edit the best answer to make it correct now. So now you are doing SFT when all the answer was really bad, so that you can get out from the local minimum of your model. I think this is like super promising and it seems like there's just, well, do you have an idea? You know, you started with this question of how much scale you need, do you now have a better idea?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.