Dwarkesh Patel: uncertainty

Transcript context

…Even within Blackwell, there are differences in performance when you’re communicating on the chip versus across the chips. Those bounds are obviously much smaller than when you’re going out of the entire chip. When you scale the number of chips up, there is some performance loss. It’s not perfect, but it is way better than different entire packages. How large can advanced packaging scale? The way Nvidia is doing it is CoWoS. Google, Broadcom, MediaTek, and Amazon’s Trainium are all doing CoWoS. But actually you can go look back at what Tesla did with Dojo, which they cancelled and restarted. Dojo was a chip that was the size of an entire wafer. They had 25 chips on it. There were some tradeoffs. They couldn’t put HBM on it. But the positive side was that they had 25 chips on it. To date, it is still probably the best chip for running convolutional neural networks. It’s just not great at transformers because the shape of the chip, the memory, the arithmetic, and all these various specifications are just not well-suited for transformers. They’re well-suited for CNNs. Dojo chips were optimized around that, and they made a bigger package. But as you make packages bigger and bigger, you have other constraints: networking speed, memory bandwidth, and cooling capabilities. All of these things start to rear their heads. It’s not simple. But yes, you will see a trend line of more chips on the package, and yes, you’re going to be able to do that on 7 nm. In fact, that’s what Huawei did with their Ascend 910C or D. They initially put one, and then they did two. They’re focusing on scaling the packaging up because that is an area where they can advance faster than process technology where they can’t shrink. But at the end of the day, that’s something you can do on the leading-edge chips too. Anything you do on 7 nm, you can also probably do on 3 nm in terms of packaging. If we end up in this world in 2030 where the West has the most advanced process technology but has not ramped it up as much, whereas China… I don’t know if you think by 2030 they would have EUV and 2 nm or whatever. But they are semiconductor-pilled and they are producing in mass quantity. Basically, I’m wondering what the year is where there’s a crossover, where our advantage in process technology has faded enough, and their advantage in scale has increased enough. And also, if their advantage in having one country with the entire supply chain indigenized—rather than having random suppliers in Germany and the Netherlands—would mean that China would be ahead in its ability to produce mass flops. To date, China still does not have an entirely indigenized semiconductor supply chain.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.