Yann LeCun: prediction

Transcript context

…So really goes to the… I think the fundamental question is can you build a really complete world model, not complete, but one that has a deep understanding of the world? Yeah. So can you build this first of all by prediction, and the answer is probably yes. Can you build it by predicting words? And the answer is most probably no, because language is very poor in terms of weak or low bandwidth if you want, there’s just not enough information there. So building world models means observing the world and understanding why the world is evolving the way it is, and then the extra component of a world model is something that can predict how the world is going to evolve as a consequence of an action you might take. So one model really is here is my idea of the state of the world at time, T, here is an action I might take. What is the predicted state of the world at time, T+1? Now that state of the world does not need to represent everything about the world, it just needs to represent enough that’s relevant for this planning of the action, but not necessarily all the details. Now, here is the problem. You’re not going to be able to do this with generative models. So a generative model has trained on video, and we’ve tried to do this for 10 years, you take a video, show a system, a piece of video, and then ask you to predict the reminder of the video, basically predict what’s going to happen. One frame at a time, do the same thing as the autoregressive LLMs do, but for video.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.