Akarsh Kumar: belief

Transcript context

…This hierarchical locking tells us something really important about how representations emerge. We think it's about finding the right building blocks now, but weirdly, it's about making future discoveries more likely. You know, a bit like how good code now reduces technical debts in the future. Bad code is the sand castle. And there's an evolutionary principle which makes sure that these superior foundations win out over time. Pick breeder, I think what's especially what was at play was like the evolution of evolvability because people are only selecting for what they wanted. Right? What the what looked good. But implicitly, there's also an implicit selection pressure for evolvable things. So if there's, like, 2 versions of the skull, which is 1 is, like, spaghetti and 1 is, like, very modular and composable, after a few generations of evolution, the 1 that's more evolvable will be the 1 that wins out. Right? Just like in natural evolution, the evolution of evolvability. And this evolvability combined with the serendipity is what I think gives you these nice representations. The thing that I think makes this really intriguing is that it gives you something that otherwise could never exist, which is a counterexample that there actually do exist networks that don't have that issue. You would think that that's just intrinsic to neural representation, that somehow they just look like kind of entangled messes, and that's just the way life is. But clearly, it's not how life has to be.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.