Lex Fridman: uncertainty

Transcript context

…Sure. And a lot of plans that people know about that are relatively high level are actually learned. Most people don’t invent the plans by themselves. We have some ability to do this of course, obviously, but most plans that people use are plans that have been trained on, they’ve seen other people use those plans or they’ve been told how to do things, right? That you can’t invent how you take a person who’s never heard of airplanes and tell them how do you go from New York to Paris? And they’re probably not going to be able to deconstruct the whole plan unless they’ve seen examples of that before. So certainly LLMs are going to be able to do this, but then how you link this from the low level of actions, that needs to be done with things like JEPA that basically lift the abstraction level of the representation without attempting to reconstruct the detail of the situation, that’s why we need JEPAs for. I would love to sort of linger on your skepticism around auto regressive LLMs. So one way I would like to test that skepticism is everything you say makes a lot of sense, but if I apply everything you said today and in general to I don’t know, 10 years ago, maybe a little bit less, no, let’s say three years ago, I wouldn’t be able to predict the success of LLMs. So does it make sense to you that autoregressive LLMs are able to be so damn good? Yes.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.