Lex Fridman: belief

Transcript context

…Yeah. So that’s why I think that there’s a set of hugely interesting questions to be asked about the ways in which large language models actually do represent the world. Because what is clear is that they’re very good at acting human. The open question in my mind is, is the easiest, most efficient, best way to act human to do the same things that human beings do or are there other ways? And I think that’s an open question. I just heard a talk by Melanie Mitchell at Santa Fe Institute, an artificial intelligence researcher, and she told two stories about two different papers, one that someone else wrote and one that her group is following up on. And they were modeling Othello. Othello, the game with a little rectangular board, white and black squares. So the experiment was the following. They fed a neural network the moves that were being made in the most symbolic form, E5 just means that, okay, you put a token down on E5. So it gives a long string, it does this for millions of games, real legitimate games. And then it asks the question, the paper asks the question, “Okay, you’ve trained it to tell what would be a legitimate next move from not a legitimate next move. Did it in its brain, in its little large language model brain.” I don’t even know if it’s technically large language model, but a deep learning network. “Did it come up with a representation of the Othello board?” Well, how do you know? And so they construct a little probe network that they insert, and you ask it, “What is it doing right at this moment?” And the answer is that the little probe network can ask, “Would this be legitimate or is this token white or black?” Or whatever, things that in practice would amount to it has invented the Othello board. And it found that the probe got the right answer, not 100% of the time, but more than by chance, substantially more than by chance. So they said there’s some tentative evidence that this neural network has discovered the Othello board just out of data, raw data. But then Melanie’s group asked the question, “Okay, are you sure that that understanding of the Othello board wasn’t built into your probe?” And what they found was at least half of the improvement was built into the probe. Not all of it. And look, a Othello board is way simpler than the world. So that’s why I just think it’s an open question, whether or not … I mean, it would be remarkable either way to learn that large language models that are good at doing what we train them to do are good because they’ve built the same kind of model of the world that we have in our minds or that they’re good despite not having that model. Either one of these is an amazing thing. I just don’t think the data are clear on which one is true. I think I have some sort of intellectual humility about the whole thing because I was humbled by several stages in the machine learning development over the past 20 years. And I just would never have predicted that LLMs, the way they’re trained, on the scale of data they’re trained would be as impressive as they are. And that’s where intellectual humility steps in, where my intuition would say something like with Melanie, where you need to be able to have very sort of concrete common sense reasoning, symbolic reasoning type things in a system in order for it to be very intelligent. But here, I’m so impressed by what it’s capable to do, train on the next token prediction essentially … My conception of the nature of intelligence is just completely, not completely, but humbled, I should say. Look, and I think that’s perfectly fair. I also was, I almost say pleasantly, but I don’t know whether it’s pleasantly or unpleasantly, but factually surprised by the recent rate of progress. Clearly some kind of phase transition percolation has happened and the improvement has been remarkable, absolutely amazing. That I have no arguments with. That doesn’t yet tell me the mechanism by which that improvement happened. Constructing a model much like a human being is clearly one possible mechanism, but part of the intellectual humility is to say maybe there are others.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.