Tim Scarfe: belief

Transcript context

…I mean, I think they can't go outside their distribution because it's just something that that type of mathematical model can't do. You know, I mean, it can do it, but it won't do it well. You know, when you look at the kind of 2 d case of fitting a curve to data, once you go outside the area that the data covers, the curves disappear off into space in wild directions, you know. And that's all we're doing, but we're doing it in multiple dimensions. Yep. I think Boden might be pretty shocked at how far compositional creativity can go when you can compose the entirety of the human knowledge corpus. And I think this is where people often get confused, because it's like So for example, I was talking to Chris Latner yesterday about how Claude Anthropic, you know, had had got Claude to write the C compiler. And they were like, oh, this is a clean room C compiler. You can tell it's clean room because it was created in Rust, you know, and so Chris created the kind of, you know, I guess it's probably the top most widely used c c plus plus compiler nowadays playing on top of LLVM, which is the most widely used kind of foundation for compilers. They're like, Chris didn't use rust. This is, you know, and we didn't give it access to any compiler source code. So it's a clean room implementation. But that misunderstands how LLMs work. Right? Which is all of Chris's work was in the training data. Many many times LLVM is used widely and lots and lots of things are built on it, including lots of c and c plus plus compilers. Converting it converting it to Rust is an interpolation between parts of the training data, you know. It's a style transfer problem. So it's definitely compositional creativity at most, if you can call it creative at all. And you actually see it when you look at the the repo that it created. It's copied parts of the LLVM code, which today Chris says like, oh, I made a mistake. I shouldn't have done it that way. Nobody else does it that way, You know? Oh, wow. Look. They're the only other 1 that did it that way. That doesn't happen accidentally. That happens because you're not actually being creative. You're actually just finding the kind of nonlinear average point in your training data between, like, Rust things and building compiler things? All of that is true. I mean, first of all, I I think we shouldn't underestimate the size of how big this combinatorial creativity is. So all of that is true. So the code is on the Internet, but also, they had a whole bunch of tests which were scaffolded, which meant that every single time some code was committed, they could run the test and they basically had a critic. And they could then do this autonomous feedback loop. So in a sense, it's very similar to the recent research by OpenAI and Gemini, where you're trying to solve a problem in math and you already have an evaluation function, the same on the ARC prize, right? You have an evaluation function. And what people discount is even knowledge of what the evaluation function is, is partial knowledge of the problem. So you can then brute force search. You can use the statistical pattern matching. Use the verifier as a constraint, and you can actually they don't even need to do that. Right? Like, they literally already know how to pass those tests because there's lots of software that already does it. Right. So it just uses that and translates them to Rust. Like, that's that's all it did, which is impressive.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.