Dwarkesh Patel: uncertainty

Transcript context

…That’s right. I don’t know if you have any commentary on what the brain might be doing versus how these chips work. Let’s take the clock speed one first. The clock speed is quite high on a chip because that drives higher throughput. When we compare a GPU running some workload, it’s running batch size 1,000. Whereas the brain is not running batch size 1,000, there’s only one of me. You could imagine saying, “Take a GPU and instead of running at a gigahertz, run it at a megahertz,” and that would start to look a little more like the equivalent things you’re talking about in the brain. But in the way silicon works, that does not give you a 1,000x advantage in energy efficiency. What it ends up looking like is you just run this circuit once to stabilization, and then it will sit idle for a long period of time. It doesn’t consume a lot of energy while it’s sitting idle because most of the energy is consumed in toggling bits from zero to one and back. Let’s talk about the energy consumption of a circuit like this. The way to think of a bit being stored is that you’ve deposited some charge in a capacitor sitting somewhere in the chip implicitly. It becomes charged when the bit becomes a one, and then it becomes discharged when it next goes to a zero. That cycle of charging the capacitor and then dumping that charge out to ground is where the energy is consumed. This is called the dynamic or switching power, and it’s most of the energy consumption of a chip. There is some other energy consumption just coming from the fact that insulators aren’t perfect, but we’ll discard that. Most of the energy consumption comes from toggling from zero to one and back to zero. If you run a chip much slower and you only clock it once every thousand clock cycles, you will have 1,000 times fewer transitions. It will be about 1,000 times less energy consumption. But it’s not a substantial advantage in energy efficiency.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.