Evidence receipt / belief
Published · transcript-backedSpeaker unverified: belief
27 Sept 2025 Machine Learning Street Talk New top score on ARC-AGI-2-pub (29.4%) - Jeremy Berman
“Will be will be logarithmic. So I wouldn't expect using these basically, using the language models we have today, I would not expect anyone to break, let's say, 40%, but you could probably make my solution twice as efficient, I would say.”
— Speaker unverified
Source trail
Everything needed to verify it.
- Speaker
- Speaker unverified
- Attribution
- Not verified from this transcript
- Claim type
- belief
- Recorded
- 27 Sept 2025
- Publisher
- Machine Learning Street Talk
Transcript context
…Yeah. I I just wonder how close do you think are we getting to the kind of Pareto optimal of of this approach. I mean, to give you a few examples, we interviewed the Alpha Revolve team. That was fascinating. And maybe you can contrast with with those guys. Surcana AI yesterday. Yeah. Robert Lange was the first author. They've released this. I think it's called Shrinker. And that was a kind of similar kind of evolutionary, you know, program thing. And and they had some cool features in there like, you know, using Bandits and using UCB. And I I guess, like, are we getting to the point where we're gonna really figure out what is the most optimal way to do this? By way, they were also switching between different foundation models. I think improvements will be log. Right? Will be will be logarithmic. So I wouldn't expect using these basically, using the language models we have today, I would not expect anyone to break, let's say, 40%, but you could probably make my solution twice as efficient, I would say. You wouldn't get such so much about yeah. You wouldn't get, you wouldn't get more than a few percentage points more accurate is my guess, but you could make it a lot more efficient. There's a ton of efficiency gains to be made. We've been dancing around this a little bit that, you know, Cholet's measure of intelligence was all about resisting memorization. And there is this question now, you know, which is to what extent are we actually building systems that we might call intelligent? And he says that intelligence is simply like the efficiency of knowledge acquisition. And and I I'm really on board with that. And and I I think it's fair to say at the moment that, let's say, like, you know, your solution and Greenbat solution, it's quite ephemeral and stateless, which is to say that when you have a new task come along, you kind of start again from scratch, which means it's not really like adapting and acquiring new knowledge and transferring that knowledge. So maybe you would agree that in the spirit of Charle's measure of intelligence at at at the moment, it's more of a kind of searching approach. But what do you think we would need to do to kind of, you know, make it more adaptable?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.