High Signal Podcasts Evidence ledger
Method
Browse
← Back to evidence

Evidence receipt / commitment

Published · transcript-backed

Tim Scarfe: commitment

23 Nov 2025 Machine Learning Street Talk He Co-Invented the Transformer. Now: Continuous Thought Machines - Llion Jones and Luke Darlow [Sakana AI]

“You know, there's this path dependence idea. So we need to do supervision because we have the path dependence so we can guide the generation of the language models.”

— Tim Scarfe

Source trail

Everything needed to verify it.

Speaker
Tim Scarfe
Attribution
Verified speaker
Claim type
commitment
Recorded
23 Nov 2025
Publisher
Machine Learning Street Talk

Transcript context

…I think so. Whether or not that's going to replace us completely, I go back and forth on. Powerful algorithms are finding helping us do research. Right? So I think it might just end up being a more powerful version of that. Right? So I I know the the AI scientists that we we released, we showed that you could actually go end to end. Right? Go from seeding the system with an idea for a research paper, and then just take your hands off, and just let it go. Think about the idea, write the code, run the code, collect the results, and write the paper, to the point that we were actually able to get it to a 100% AI generated paper accepted to to a workshop recently. Right? But I think we did that to show that you could do it, right, as a sort of demonstration. In a real system, I think I would want it to be much more interactive. Right? I would wanna be able to see it with an idea and then have it come back with more ideas, have a discussion with me, then go away to write the code. Want to look at the code and check it, and then discuss the results as they're coming out. So that's the sort of near term future that I that I would envision or how I would like to do research with an AI. And could you introspect on that? Is it because you feel we need supervision because the models don't yet understand? You know, there's this path dependence idea. So we need to do supervision because we have the path dependence so we can guide the generation of the language models. Maybe in the future the language models will just understand better themselves. But there's also the output dimension, which is that we want to produce artifacts that extend the phylogeny of human interest. We want it to be human relevant. Yeah. I I think it's more that, you know, in that initial seed idea, it's probably impossible to actually describe exactly what you want. It's exactly the same with, you know, when I have an intern. I can't just have an intern come into the company, and I go, I have this mad idea, and then just explain it to them, and then just leave them alone for 4 months. There's a back and forth, because I have a particular idea that I want to explore, and I need to keep steering them in the direction that's I I, you know, that I had in my my mind originally. So I think it's more like that, basically.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.

Search evidence