Tim Scarfe: belief

Transcript context

…I think that would be a terrible, terrible idea. Like, you don't wanna remove people from their exploratory environment, you know? Research and science is developed by people building insight, you know? Whoever you listen to, you know, whether it be Feynman or whatever, like, you always hear from the great scientists how they build deeper intuition by by building mental models, which they get over time, by interacting with the things that they're learning about. And like in Feynman's case, because it was theoretical physics, he couldn't actually pick up spinning quark, but he did literally study spinning plates, you know. You gotta find ways to to deeply interact with with what you're working with. Like, so so many times I've seen data science teams because you're right. Data science teams aren't very familiar with Git and aren't very familiar with things that they do need to understand. And so often, I've seen a software engineer will become their manager, and their fix to this will be to tell them all to stop using Jupyter notebooks. And now they have to use all these reproducible blah blah virtual, you know, virtual end blah blah. They destroy these teams over and over again. I've seen this keep happening. Because the solution is not create more discipline and bureaucracy. It's solve the actual problem. So for example, we we built a thing called an n b merge driver, which so a lot of people don't realize this, but actually notebooks are extremely git friendly. It's just that git doesn't ship with a merge driver for them. So git only ships with a merge driver for line based text files. But it's fully pluggable. And so you can easily plug in 1 for JSON files instead. And so we wrote 1. So now when you diff, you know, when you get a git diff with our merge driver, you see cell level diffs. If you get a merge conflict, get search level cell level merge conflicts. The notebook is always openable in Jupyter. NBDime did the same thing, so 2 independent implementations of this. So yeah, there were problems to solve, you know, but the solution to it was not throw away Brett Victor's ideas and make people further away from from their exploratory tools. But to fix the exploratory tools, and I think all software developers should be using exploratory based programming to deepen their understanding of what they're working with. So that they end up with a really strong mental model of the system that they're building and they're working with. And then they can come up with better solutions, more incrementally, better tested. I basically never have to use a debugger, because I basically never have bugs. And it's not because I'm a particularly good programmer, it's because I build things up small little steps, and each step works and I can see it working and I can interact with it. So there's no room for bugs, you know? You know, I'm so torn on this because I agree with you. And I'm also skeptical of people who say that organizations, they they they converge onto ways of doing things, they no longer need to evolve. They no longer need to adapt. Innovation is adaptivity, right? And we should increase the surface area of adaptivity as much as we possibly can. So we need people that are constantly testing new ideas, finding these constraints. But by the same token, we need to use the cloud, we need to use CICD. We need to get this stuff into production. Yeah. So do you do you do but, like, there's absolutely no like, so NB dev ships with out of the box CI integration, and that's like the tests are literally there, like, because the source is a notebook, the entire exploration of like, how does this API work, you know, what does it look like when you call it, The implementation of the functions, the examples of them, the documentation of them, the tests of them are in 1 place. So it's much easier to be a good software engineer in this environment. So, yeah, like, do do both, you know.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.