Trenton Bricken: belief

Transcript context

…If RL requires a bunch of inference to do the training in the first place, does that push against the thing you were talking about where we actually need a bigger model in order to have brain-like energy? But then also it's more expensive to train it in RL. So, where does that balance out? I think we got to drink the bitter lesson here. Yeah, there aren't infinite shortcuts. You do just have to scale and have a bigger model, and pay more inference for it. If you want AGI, then that's what you got to pay the price of. But there's a tradeoff equation here. There is science to do which everyone is doing. What is the optimal point at which to do RL? Because you need something which can both learn, and discover the sparse reward itself. So you don't want a one parameter model. Useless, even though you can run it really fast. You also don't want a 100T model. It's super slow. The marginal benefit of its learning efficiency is not worth it. So, there's a Pareto frontier here. What's the optimal model size of your current class of capabilities, and your current set of RL environments, and this kind of stuff.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.