Sergey Levine: evaluation

Transcript context

…We already have LLMs which are broadly deployed. That hasn't resulted in some sort of flywheel, at least not some obvious flywheel for the model companies where now Claude is learning how to do every single job in the economy or GPT's learning how to do every single job in the economy. So, why doesn’t that flywheel work for LLMs? Well, I think it's actually very close to working and I am 100% certain that many organizations are working on exactly this. In fact, arguably there is already a flywheel. It’s not an automated flywheel but a human-in-the-loop flywheel. Everybody who's deploying an LLM is of course going to look at what it's doing and it's going to use that to then modify its behavior. It's complex because it comes back to this question of representations and figuring out the right way to derive supervision signals and ground those supervision signals in the behavior of the system so that it improves on what you want. I don't think that's a profoundly impossible problem. It's just something where the details get pretty gnarly and challenges with algorithms and stability become pretty complex. It's something that's taken a while for the community collectively to get their hands on. Do you think it'll be easier for robotics? Or do you think that with these kinds of techniques to label data that you collect out in the world and use it as a reward, the whole wave will rise and robotics will rise as well? Or is there some reason robotics will benefit more from this?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.