Jeremy Howard: evaluation

Transcript context

…I agree with all of that. But then there's the other interesting angle, which is that there was a famous talk by Joel Grus, and we'll talk about this. And and he said that notebooks are terrible. They're they're really bad from a software engineering point of view. And and at the time, and maybe still now to a certain extent, I I agree with him because, you know, I've I've I've done ML DevOps. I've worked in large organizations, you know, like trying to figure out how do we bridge like data science and software engineering. And Claude code is already more towards the software engineering side and what that means is it creates idempotent, stateless, repeatable artifacts, right? So as you say, from a pedagogical point of view, it's really good having this stateful feedback because I can understand what's going on. But then I need to translate that into something which is deployable. And Can you tell us the story of you you you responded to Joel Grus, didn't you? And and it was a bit of a fiasco, wasn't it? But what just just tell us about that story. He did a really good video called I don't like notebooks. It was hilarious. It was really well done. And, yeah, I was totally wrong. And all the things he said notebooks can't do, they can. And all the things he said you can't do with notebooks, I do with notebooks all the time. So it was a very good, amusing incorrect talk. So then I did a kind of a parody of it called I like notebooks, in which I basically copied with credit most of his slides and showed how every 1 of them was totally incorrect. But, like, I actually think your comment about it does come down to the heart of it, which is this this this difference between, like, how software engineering is normally done versus how scientific research and similar things is normally done. And I think and I agree there is a dichotomy there. And I think that dichotomy is a real shame because I think software development is being done wrong. It's being done in this way, which is, yeah, all about reproducibility and these like dead these dead pieces, you know. It's all dead code, dead files. I will never be able to express this 1 millionth as clearly as Brett Victor has in his work, so I'd encourage people who haven't watched Brett Victor to to watch him. But, you know, he he shows again and again how a direct connection, you know, a direct visceral connection with the thing you're doing is is all that matters, you know. And that's his mission, is to make sure people have that connection. And that's basically my mission as well. So for me, traditional software engineering is as far from that as it is possible to get. I think it's I think it's gross. Like, I I I find it disgusting. And I find it sad that people are being forced to work like that. It's like, I think it's inhumane. And I just don't think it works very well. I mean, empirically, it doesn't work very well. And it's much less good for for AI as well, as it's much less good for humans. It hasn't always been that way. Like, you know, with with Alan Kay and Smalltalk and Iverson and APL, you know, Lisp Wolfram with Mathematica. To me, these were the golden days when when people were focused on the question of how do we get the human into the computer to work as closely with it as possible. You know, that's where the the mouse came from, for example. I've got to like click and drag and visualize entities in your computer as things you can move around. So I feel like we've lost that. I think it's really sad. Yeah. With Claude code and stuff, the the default way of working with them is to go super deep into it. It's like, okay, there's a whole folder full of files, you never even look at them, your entire interaction with it is through a prompt. Yeah.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.