Adam Marblestone: evaluation

Transcript context

…But it never got supervision on that. So how does it…? Well, it’s because the Learning Subsystem is a powerful learning algorithm that does have generalization, that is capable of generalization. The Steering Subsystem, these are the innate responses. You’re going to have some built into your Steering Subsystem, these lower brain areas: hypothalamus, brainstem, et cetera. Again, they have their own primitive sensory systems. So there may be an innate response. If I see something that’s moving fast toward my body that I didn’t previously see was there and is small and dark and high contrast, that might be an insect skittering onto my body. I am going to flinch. There are these innate responses. There’s going to be some group of neurons, let’s say, in the hypothalamus, that is the I-am-flinching or I-just-flinched neurons in the hypothalamus. When you flinch, first of all, it’s a negative contribution to the reward function. You didn’t want that to happen, perhaps. But that’s a reward function that doesn’t have any generalization in it. I’m going to avoid that exact situation of the thing skittering toward me. Maybe I’m going to avoid some actions that lead to the thing skittering. That’s a generalization you can get, what Steve calls downstream of the reward function. I’m going to avoid the situation where the spider was skittering toward me, but you’re also going to do something else. There’s going to be a part of your amygdala, say, that is saying, “Okay, a few milliseconds, hundreds of milliseconds or seconds earlier, could I have predicted that flinching response?” It’s going to be a group of neurons that is essentially a classifier of, “Am I about to flinch?” And I’m going to have classifiers for that for every important Steering Subsystem variable that evolution needs to take care of. Am I about to flinch? Am I talking to a friend? Should I laugh now? Is the friend high status? Whatever variables the hypothalamus, brainstem, contains… Am I about to taste salt? It’s going to have all these variables and for each one it’s going to have a predictor. It’s going to train that predictor. Now the predictor that it trains, that can have some generalization. The reason it can have some generalization is because it just has a totally different input. Its input data might be things like the word “spider”, but the word “spider” can activate in all sorts of situations that lead to the word “spider” activating in your world model. If you have a complex world model with really complex features that inherently gives you some generalization. It’s not just the thing skittering toward me, it’s even the word “spider” or the concept of “spider” is going to cause that to trigger. This predictor can learn that. Whatever spider neurons are in my world model, which could even be a book about spiders or somewhere, a room where there are spiders or whatever that is… The amount of heebie-jeebies that this conversation is eliciting in the audience……

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.