Eric Jang: belief

Transcript context

…Tell me if this is the wrong way to think about it. When I learn how an LLM works and how simple RLVR is as an algorithm, I’m stunned by the kinds of things it can do. It can learn how to build very complicated code repositories simply from getting a yes or no. Here, if you understand it more deeply, just predicting MCTS, AlphaGo seems less impressive in retrospect the more you understand it. You’re putting in a lot of bias by telling it how it should titrate exploration as things go on. You’re building this very explicit tree search for it. I don’t know if you share that intuition where the more you understand it, the less impressive the accomplishment in 2017 seems. I personally disagree. I think they’re profound for different reasons. I don’t understand the LLM RL enough to comment on it on your podcast. But why is AlphaGo a profound accomplishment? It’s worth stepping back a little bit. It is different from modern RL, and we can talk a little bit about some of the algorithmic choices there. I think the most profound thing here is that a 10-layer neural network pass, basically 10 steps of reasoning… Of course, the reasoning is not just one trail of thought. It could be distributed representations and a lot of thoughts going on at the same time. But by construction, let’s say a 10-layer neural network can only do 10 sequential steps of thinking. 10 steps of neural network parallelized distributed-representation thinking is able to amortize and approximate to very high fidelity a nearly intractable search problem. This was a breakthrough that I think most people don’t even fully comprehend today, how profound that accomplishment is. This is what also girds AlphaFold, for example, where you have a very, very difficult physical simulation process where you would need to roll out as so many microscale simulations, and yet 10 steps of a somewhat small neural network can somehow capture what feels like an NP-class problem into a single problem. It actually makes me wonder if our understanding of problems like P=NP, or these fundamental computational hardness problems, is incomplete. Obviously, it’s not a proof of P=NP, but there’s something to it that is very disturbing, where what felt like a very hard problem can fall to a very simple macroscopic solution. That is a very interesting insight, that a lot of problems which are proven to be NP-hard—I don’t know if Go is proven to be NP-hard, but protein folding, et cetera—neural networks can solve. They’re NP-hard in the worst case, but we’re usually not concerned about the worst case. These problems usually have a lot of structure to them.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.