Demis Hassabis: evaluation

Transcript context

…I have two questions based on that. With AlphaGo, you had a very concrete win condition: at the end of the day, do I win this game of Go or not? You can reinforce on that. When you’re thinking of an LLM putting out thought, do you think there will be this ability to discriminate in the end, whether that was a good thing to reward or not? Of course that’s why we pioneered, and what DeepMind is sort of famous for, using games as a proving ground. That’s partly because it’s efficient to research in that domain. The other reason is, obviously, it’s extremely easy to specify a reward function. Winning the game or improving the score, something like that is built into most games. So that is one of the challenges of real-world systems. How does one define the right objective function, the right reward function, and the right goals? How does one specify them in a general way, but specific enough that one actually points the system in the right direction? For real-world problems, that can be a lot harder. But actually, if you think about it in even scientific problems, there are usually ways that you can specify the goal that you’re after. When you think about human intelligence, you were just saying that humans thinking about these thoughts are just super sample-efficient. Einstein coming up with relativity, right? There’s thousands of possible permutations of the equations. Do you think it’s also this sense of different heuristics like, “I’m going to try out this approach instead of this”? Or is it a totally different way of approaching and coming up with that solution than what AlphaGo does to plan the next move?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.