← All source episodes Machine Learning Street Talk / episode intelligence
The Fractured Entangled Representation Hypothesis (Kenneth Stanley, Akarsh Kumar)
6 Jul 2025 34 published claims 4 attributable people
Speakers in the public record
Claim mix
belief 20evaluation 7uncertainty 4preference 2commitment 1
Evidence policy
Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.
Claim ledger
The useful parts, with receipts.
34 published records
“You know, it it it feels to me that the missing link is having the correct level of abstraction and being able to do this iterative open ended search. They can't do that because they simply don't have the abstractions.”
- Publisher
- Machine Learning Street Talk
“These are continuous functions because most of the time in a neural network, like, if you if you give it a test sample, which is outside of the training support, it you're in no man's land.”
- Publisher
- Machine Learning Street Talk
“Duggar. But some people say, I don't know, like polysemanticity or grokking or scale and, you know, that that it just it just appears like the neural network isn't grokking it.”
- Publisher
- Machine Learning Street Talk
“I think 1 of the really interesting things about the observation in this paper is it pokes a hole, I think, in a very deep assumption that we have that if the results are good, then what's underneath the hood is also good.”
- Publisher
- Machine Learning Street Talk
“I mean, there's there's a few points here because I guess like what I was, where I was going with this before is, you take y equals x squared, and the reason why we think of it as robust is for any value of y, it kind of does something it does something reasonable.”
- Publisher
- Machine Learning Street Talk
“Like like, I think, you know, it's interesting, like, in the field of AI that if you go back 10 years or so, like, most of the interactions of AIs with dynamic training environments would be in simulations.”
- Publisher
- Machine Learning Street Talk
“I think about is, like, it's like a memory with an algorithm. You want to use as much as necessary and no more.”
- Publisher
- Machine Learning Street Talk
“I think there are other aspects to natural evolution, 1 of these being the constraint to use less resources, less energy, simplicity of solution.”
- Publisher
- Machine Learning Street Talk
“I think Ken would agree with the statement that if we really were to understand it, then we should have, then there should be exist like an algorithm that we can scale up right now that can recreate all the glamour of evolution.”
- Publisher
- Machine Learning Street Talk
“The point is that the things that we care about are actually the side effects of the constraint.”
- Publisher
- Machine Learning Street Talk
“We know that we have certain things we can compose, and we know that we can compose them in certain topologies, and we know that invariably if we follow that trajectory, we will land on interesting things, even though we don't necessarily know exactly what we will land on.”
- Publisher
- Machine Learning Street Talk
“We could easily discover something that what that wipes YouTube away. So it's about this epistemic gap between what we really want and what we think we want.”
- Publisher
- Machine Learning Street Talk
“I mean it's it's brilliant, it's insightful, it's important, It's visually 1 of the most beautiful papers I've seen in a in a long time. And I think even people just looking at this, will get the insight that you're talking about right now, like with Einstein, you know, because I mean, sure, we're not Einstein, but I think anybody who introspects the way they think, the way they think about solving problems, the way they think about a skull, the way they think about an apple, you know, they're gonna find that the images in this paper completely reflect the way we think about the world, the way we model the world.”
- Publisher
- Machine Learning Street Talk
“Like, you know, we we think about, you know, when you say, like, eventually, if you experience enough of the world, like you say, of course, you might expect that your representation start to mirror just the way that the world is.”
- Publisher
- Machine Learning Street Talk
“I mean, 1 of the really important points I wanna add to this, this may also apply to humans. So I don't want to seem like I'm saying that all humans have unbelievably beautiful, unfractured We also, I think, victims of going through things in a bad order sometimes.”
- Publisher
- Machine Learning Street Talk
“I mean, like, creating an open end algorithm which solves this issue, I think, Jeff Kloon on our paper, he calls it, like, the trillion dollar question or the trillion dollar algorithm because that's basically, like, how you, if you think that the UFR, like, the unified factor representations are akin to, like, a human, then that's basically, like, creating, like, a human representation.”
- Publisher
- Machine Learning Street Talk
“I think if you still found a way to do this kind of evolutionary building up from simpler, you know, mappings, what you would end up doing is there would be there would be kind of an a a simple higher order layer that say took the spiral and chopped it up into 4 quadrants, like nice nice quadrants.”
- Publisher
- Machine Learning Street Talk
“What we care about is the all the downstream stuff that we're gonna use it for later, which is much harder to quantify, much harder to formalize than just a training loss.”
- Publisher
- Machine Learning Street Talk
“You could say, this is good enough, I'm happy. But I think in terms of what we can't do because of that, that we might someday in the future be able to do, The things we can't do are the things where the human mind isn't having the ideas.”
- Publisher
- Machine Learning Street Talk
“I think, you know, your communication of open ended search and and kind of these this take on it has has been sharpened, you know, significantly since like 4 years ago when we talked.”
- Publisher
- Machine Learning Street Talk
“I suspect it has more implications than just that because when we go beyond efficiency to things like creativity, what you're seeing is that the dimensions that have been discovered in the skull for example, align with new skulls, imagining new things in the world.”
- Publisher
- Machine Learning Street Talk
“I think there's a a direct or deep connection with, you know, poet like the work, you know, your earlier paper, right, on this kind of increasingly complex curriculum and environment where you start off training in simple cases and make them more and more complex.”
- Publisher
- Machine Learning Street Talk
“I think about 4 years ago that we interviewed Kenneth the first time and it was eye opening then and and it and, you know, it's it's opened a lot of great open ended exploration for me personally.”
- Publisher
- Machine Learning Street Talk
“Like, what are you objecting to? But the but the point is that it can still be an impostor because it's like what we care about here is not just that it's going to get answers right, like get good test scores, like seem to be plausibly human when you talk about things that are in distribution.”
- Publisher
- Machine Learning Street Talk
“I like that Goldilocks analogy a lot for evolution because I think Joel and Risto had a paper where it's like you need, like, some sort of catastrophic events to happen in order to get adaptable solutions.”
- Publisher
- Machine Learning Street Talk
“We also show new examples. But you you know, the word reason I use the word indirect is because unlike these pick breeder images, we can't just go in and look at a neuron and know what it does explicitly because that's what's so nice about pick breeder images because they're 2 d.”
- Publisher
- Machine Learning Street Talk
“It doesn't work autonomously. It's not creative because it's not built on the foundation of a representation that describes the world well.”
- Publisher
- Machine Learning Street Talk
“Right? And we are different from that because, you know, the the very basis of how we think is correlated to how the world works.”
- Publisher
- Machine Learning Street Talk
“It could be though something implicit and indirect. And so I think I think overall, it's it's like Akash said, there's still some unknowns here about what actually what actually matters, what doesn't matter, especially with respect to the representation.”
- Publisher
- Machine Learning Street Talk
“Like, I I I we don't know because everybody's just stuck on SGD SGD scale scale scale is enough.”
- Publisher
- Machine Learning Street Talk
“And he was like, I don't know. And I don't I don't wanna give you a recommendation because the point is that none of us are we're supposed to all follow what we think is the coolest thing and not just like just decide like, oh, when you put a 100% of our resources into this 1 thing.”
- Publisher
- Machine Learning Street Talk
“Like not butterflies, dragonflies. And and therein is the crux of the problem because we did this open ended cool thing and I mean part of the, let's say, downside of an open ended search is you don't know where you're gonna end up.”
- Publisher
- Machine Learning Street Talk
“It's just emergent from how SGD climbs these gradients. But the thing that I think makes this really intriguing, the reason that the paper is because beyond just that, is that it gives you something that otherwise could never exist, which is a counterexample, that there actually do exist networks that don't have that issue.”
- Publisher
- Machine Learning Street Talk
“That's a that I think maybe that's a bit unknown for me. I'm not sure because I think there's there's trade offs there.”
- Publisher
- Machine Learning Street Talk