Reiner Pope: evaluation

Transcript context

…What does it want? The term actually, I think, originates in maybe high-frequency trading in areas like that, which is, I haven't worked in. I've just, I like reading about the software that people have built from there. It's like, for them, what does the machine want? It wants a lot of instruction-level parallelism. This is CPUs, not TPUs. Once a lot don't branch, so unpredictable branches kill your performance. Think about the things that CPUs do and how to use them best. Can I get to peak performance on a CPU? It's that idea. I think the whole idea of peak performance on a CPU is kind of crazy. No one even says, "What is peak performance? What is my percentage of peak on a CPU? " Because the performance of software running on CPUs is really bad, but running on GPUs, or TPUs, or AI chips in general, actually that is the main focus. It's like, "What is my percentage of peak? Can I get 70% or 80%?" I feel like many people listening to this know that GPUs perform better for AI workloads than CPUs. It's a funny history when you think about it, where just one day we woke up with all these very mathematically intensive workloads, first crypto mining, and then AI, so then NVIDIA is extremely well-positioned because they've been making GPUs for gamers that you would plug into... You'd buy your Dell PC back in the day and maybe upgrade the graphics card by plugging in a better NVIDIA graphics card than the one the stock Dell computer came with. They were incredibly well-positioned to capture that. I think people know that. What is the intuitive explanation as to why GPUs are better for AI workloads than CPUs? People say, "They're better for these mathematical computations." But that's a tautological answer, basically. Is there some way you can have a mental model for why that is the case? Because software instruction sets also involve doing math.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.