Speaker unverified: prediction

Transcript context

…How related is this to tool use? Are you teaching it to use tools to augment the model or are you saying, do active learning, where it's weak, go augment the model with extra data and then memorize that new data? What I said is more like in terms of directions, not for LLM 3, but when it knows how to use a tool and correct itself, this is a very promising direction that goes much beyond augmentation in the future. To keep collecting new data and new tokens, people are saying we are lacking of tokens, but if you think about those kinds of tokens, where the model always goes to correct its own weakness, it can say, that's 10 plus 10, that's an easy example, probably the model knows, but imagine for something more complex, 10 plus 10, I expect this to be 20. Let's verify with a calculator, which is easy for a basic agent now, powered by LLM. And then you verified with respect to what you expected, that it's correct. If it's not, you can back propagate this example directly to the weights and so they will keep learning new things. It makes sense. What have been your insights? You know, you mentioned about just like using calculators. What have been your insights? I think just in general, a lot of that is just driven using code generation and apart from just tool use. What have been your insights on just like the data mix of how much code, how much multilinguality, which is something that you're also passionate about? We know that that's changed between LLM 2 and LLM 3. Is it changing for different stages between the different sizes of LLM 3? Like, you know, anything like of that sort?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.