Dwarkesh Patel: belief

Transcript context

…The issue is that when I’ve got a series of instructions, if I have a branch, the actual step of processing an instruction takes a really long time. It takes maybe five nanoseconds. The time to notice that I’ve got a branch, evaluate whether the Boolean is true, update the program counter to the new target, and then read from the instruction memory could take five nanoseconds to finish. So in reality, this may finish somewhere down here. I want to run a clock speed that is much faster than what five nanoseconds allows. Five nanoseconds is a 200 MHz clock speed. I would like to run at one or two gigahertz. So I need to run other instructions while the branch is being evaluated. I just want to keep running the instructions that happen after me. But that might have been wrong. If the branch ended up being taken, then I need to know that instead of evaluating these instructions, I actually need to jump to wherever the target is and run those instructions instead. The purpose of the branch predictor is to predict, five cycles earlier, that a branch is going to happen, before you even get to that instruction. If I think about how the brain works versus what you’re describing here, at a high level the differences might be that while you can do structured sparsity in these accelerators and save yourself some area that you would have otherwise had to dedicate to gates, in the brain there’s unstructured sparsity. Any neuron can connect to any other neuron, and not in ways where they have the column aligned. Then there’s the fact that memory and compute are co-located. Although I guess you could say in a way the memory and compute are co-located on these dies too. This is exactly the co-location, in some sense, of the memory and compute.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.