Charan Ranganath: evaluation

Transcript context

…Well, it’s all interconnected. I mean, just even the thing you’ve mentioned is the moment, if we record a moment, it is difficult to express concretely what a moment is, how deeply connected it’s to the entirety of it. Maybe to record a moment, you have to make a universe from scratch. You have to include everything. You have to include all the emotions involved, all the context, all the things that built around it, all the social connections, all the visual experiences, all the sensory experience, all of that, all the history that came before that moment is built on. And we somehow take all of that and we compress it, and keep the useful parts and then integrate it into the whole thing, into our whole narrative. And then each individual has their own little version of that narrative, and then we collide in a social way, and we adjust it. And we evolve. Yeah. Yeah. I mean, well, even if we want to go super simple, like Tyler Bonin, who’s a postdoc, who’s collaborating with me, he actually studied a lot of computer vision at Stanford. And so, one of the things he was interested in is some people who have brain damage in areas of the brain that were thought to be important for memory, but they also seem to have some perception problems with particular kinds of object perception. And this is super controversial, and some people found this effect, some didn’t. And he went back to computer vision and he said, “Let’s take the best state-of-the-art computer vision models, and let’s give them the same kinds of perception tests that we were giving to these people.” And then he would find the images where the computer vision models would just struggle, and you’d find that they just didn’t do well. Even if you add more parameters, you add more layers on and on and on. It doesn’t help. The architecture didn’t matter. It was just there, the problem. And then, he found those were the exact ones where these humans with particular damage to this area called the perirhinal cortex, that was where they were struggling. So somehow this brain area was important for being able to do these things that were adversarial to these computer vision models. So then he found that it only happened if people had enough time, they could make those discriminations, but without enough time if they just get a glance, they’re just like the computer vision models. So then what he started to say was, “Well, maybe let’s look at people’s eyes.” So computer vision model sees every pixel all at once, and we don’t, we never see every pixel all at once. Even if I’m looking at a screen with pixels, I’m not seeing every pixel at once. I’m grabbing little points on the screen by moving my eyes around, and getting a very high resolution picture of what I’m focusing on, and kind of a lower resolution information about everything else. But I’m not necessarily choosing, but I’m directing that exploration, and allowing people to move their eyes and integrate that information gave them something that the computer vision models weren’t able to do. So somehow integrating information across time and getting less information at each step gave you more out of the process. The process of allocating attention across time seems to be a really important process. Even the breakthroughs that you get with machine learning mostly has to do attention is all you need, is about attention. Transform is about attention. So attention is a really interesting one. But then, yeah, how you allocate that attention, again is at the core of what it means to be intelligent, what it means to process the world, integrate all the important things, discard all the unimportant things. Attention is at the core of it, it’s probably at the core of memory too. There’s so much sensory information. There’s so much going on, there’s so much going on. To filter it down to almost nothing and just keep those parts, and to keep those parts, and then whenever there’s an error to adjust the model, such that you can allocate attention even better to new things that would resolve, maybe maximize the chance of confirming the model, or disconfirming the model that you have, and adjusting it since then. Yeah, attention is a weird one. I was always fascinated. I mean, I got a chance to study peripheral vision for a bit and indirectly study attention through that. And it’s just fascinating how good humans are looking around and gathering information.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.