Lex Fridman: preference

Transcript context

…Yes. And in all that, we don’t even respect the calling convention of the operating system in order to be faster, because we know that we are going to be called from within our binary, so we can share data without saving all the registry in the common way, because that can lead to loading and saving registry on the L1 and L2 CPU and gets us faster. So that’s why I said that understanding CPU architecture, computer architecture is key. And this is also why it’s handwritten. I don’t know anyone, I’ve never heard any other project than Dav1d doing that. This is why Kieran calls it an art, right? It is an art. I think in a mass world, there isn’t something on billions of devices. I know there are some specialist industries. I know in high-frequency trading, they take this really seriously, where they’re receiving feeds from a market, and they need to react within X number of microseconds, and so the instructions matter. But that’s not a mass-produced thing that’s on a billion devices. That’s hyper-specialized, running on hyper-specialized hardware. We’re running on all hardware from- Sorry to linger on it, but, like, that’s a really counterintuitive, almost, like, revolutionary idea here, that there’s a huge amount of value to assembly. Like, what are we supposed to take away from that? Like, what… You know, there’s a bunch of people listening to this, they’re basically like, sorry, for myself included, you know, I programmed for many, many years in C/C++, going up the standards of C++, fell in love with C++, even meta programming and so on, and then transitioned more and more because of machine learning about 15 years ago to Python. And so, like, for me in this Python world, JavaScript world, now vibe coding, where I’m just using natural language, sitting in my jacuzzi, drinking a drink… and just talking to the computer, like record stops. Why is the value to go back all the way down to the low level? Like, what’s the intuition? Because you can get more power per dollar invested, right? And sometimes it’s going to be a problem that is limited by your hardware. A good analogy is what you see in quantization in LLMs, right? And people are doing, “Oh, I’m going to do that in FP8 or FP4 or some crazy things like Microsoft Phi, who did it in 1.5,” because you’re constrained by memory, because you’re constrained by the machine you can run. Because at some point we are doing real time, and I believe this is going to happen on AI inference also, is that at some point you need to get faster, and you cannot always get more powerful hardware, right? So you need to analyze code and see where, like, where is the mission critical, where are the things that are called nonstops. And for example, dav1d is a good example. It’s going to be run billions of hours per day. That makes sense. It doesn’t make sense to be on the glue of FFmpeg- … CLI. It makes sense over there.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.