Tim Scarfe: evaluation

Transcript context

…I am Matthew Wyatt. I'm a full professor at John Hopkins University in The US and at EPFL in Switzerland. You know, those machines that can build new images that we've never seen before or say new sentences that were never heard before. Our brain seems to, you know, to to learn languages with 100,000 times less words than machines. Why is it so? Are we doing the wrong thing? And I'm very interested in, you know, should we predict in token space at a very low level or more should we train machine to predict abstractions. And so what we've been doing over the years is trying to build a framework based on physics that's really try to answer those different questions in a unified manner. Chomsky gave this poverty of stimulus argument arguing that it was actually impossible to learn to become creative from example. But if you have a deep architecture, there is a huge implicit bias to build those coarse grain variables. And so if you think about LLMs or diffusion models, the way they breed concepts, they emerge from statistics alone, just abstraction. They emerge out there in the data, they emerge. And those concept emerge if you group together configuration that predict similar context around them. And this is very pertinent because you've got a paper out basically saying that we should predict in the latent space, not the token space. So again, in those models, what we found is that those algorithms that are introspective that learn from their own latent are much more powerful in terms of sample complexity. They will eventually they will learn the same abstraction but much faster. If you never do mistakes, maybe it's a sign that you're staying a bit on the beaten path in science and some of us want to explore the jungle. In the jungle, you can be wrong. Mean, yes. Quick pause. Agents are getting smarter every day. But even the smartest agents get stuck without the right context and the right tools. That is where Notion comes in. With the recent launch of custom agents, Notion became the collaborative AI workspace where teams and agents work side by side. And now their new development platform is turning that workspace into infrastructure developers can build on. Now this is exactly how I run MLST. The whole show lives in Notion, my guests, the publishing calendar, the commercial side, everything is in there. But what's changed is that it's now agentic. I just talk to my agent, it can be Claude or any agentic harness, and then it then talks to Notion via the MCP or the CLI and it's just done and then I can access it on my phone. It's an absolute game changer. So, yeah, I'm a physicist. Actually, I really I really liked learning physics because you have to deal with, you know, nature at all possible scales. And I focused on 1 specific field in physics which is called statistical physics. And statistical physics is essentially the field where you try to understand how many entities, particles interact together to do collective phenomenon. And so a classical example is you take water, you cool down the system and at some point, boom, it freezes completely changing its organization. And so I started to work on that initially on the stock market where you have interacting agents that influence the evolution of the price, which is a very interesting sort of random walk. And then I went to study complex systems. So complex systems are physical system with a rough energy landscape. It means that if you are it's a bit like if you're flying above the Alps like you just did. If you are throwing a ball in those mountains, it could stop at many different points. So the energy landscape has many metastable states. And those systems are really intriguing. They're physical system with memory. I worked on several of those, for example, sand. So what's beautiful about sand is that it is a complex system. If you prepare 10,000 piles of sands, each of them is different. But it has intriguing again phase transition. So as you know, if you tilt a layer of sand at some point is going to flow. It means that the energy landscape was rough and you are in a metastable state but you tilted this energy landscape, you had the phase transition and then the entire system flow although it's very dense, particle managed to avoid each other. And so I've been very interested in understanding geometrically those questions. But then like 9 years ago,…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.