Dwarkesh Patel: belief

Transcript context

…ot ninety billion dollars of long-term contracts today, and they’re negotiating three-year deals today with the memory vendors. You’ve got Amazon and Google through Broadcom, Amazon directly, and AMD. These companies hold all the cards because they’ve secured the capacity. TSMC is not raising prices, but memory vendors are, to some extent, raising a lot of price. They’re going to double or triple price again, but then they’re also signing these long-term deals. Who is able to accrue all the margin dollars is potentially the cloud, potentially the chip vendors, and the memory vendors, until TSMC or ASML break out and say, “No, we’re going to charge a lot more.” But at the same time, do the model vendors get to charge crazy margins? At least this year, we’re going to see margins for the model vendors go up a lot. Because they’re so capacity constrained, they have to destroy demand. There’s no way Anthropic can continue at the current pace without destroying demand. Let’s get into logic and memory. How specifically has Nvidia been able to lock up so much of both? I think according to your numbers, by ‘27, Nvidia is going to have +70% of N3 wafer capacity, or around that area. I forget what the numbers were for memory at SK Hynix and Samsung and so forth. Think about how the neocloud business works and how Nvidia works with that, or how the RL environment business works and how Anthropic works with that. In both those cases, Nvidia is purposely trying to fracture the complementary industry to make sure that they have as much leverage as possible. They’re giving allocation to random neoclouds to make sure that there’s not one person that has all the compute. Similarly, Anthropic or OpenAI, when they’re working with the data providers, they say, “No, we’re going to just seed a huge industry of these things so that we’re not locked into any one supplier for data environments.” And I wonder why on the 3 nm process—that’s going to be Trainium 3, that’s going to be TPU v7, other accelerators potentially—why is TSMC just giving it all up to Nvidia rather than trying to fracture the market? There are a couple points here. On 3 nm, if we go back to last year, the vast majority of 3 nm was Apple. Apple is being moved to 2 nm. Memory prices are going up, so Apple’s volumes may go down. As memory prices go up, either they cut margin or they move on. There’s some time lag because they have long-term contracts, but Apple likely reduces demand or moves to 2 nm faster, where 2 nm is only capable of mobile chips today. In the future, AI chips will move there. So Apple has that. Apple is also talking to third-party vendors because they’re getting squeezed out of TSMC a little bit. TSMC’s margins on high-performance computing—HPC, AI chips, et cetera—are higher than they are for mobile, because they have a bigger advantage in HPC than they do in mobile. When you look at TSMC’s running calculus here, they’re actually providing really good allocations to companies that are doing CPUs. When you think about Amazon having Trainium and Graviton, both of those are on 3 nm, Graviton being their CPU, Trainium being their AI chip. TSMC is much more excited to give allocation to Graviton than they are to Trainium because they view the CPU business as more stable, long-term growth. As a company that is conservative and doesn’t want to ride cycles of growth too hard, you actually want to allocate to the market that is more stable with a lower growth rate first before you allocate all the incremental capacity to the fast growth rate market. That is the case generally. Same for AMD. The allocations they get on their CPUs, TSMC is much more excited about those than they are for GPUs. Likewise for Amazon. Nvidia is a bit unique because yes, they have CPUs, they make switches, they make networking, NVLink, InfiniBand, Ethernet, NICs. By and large, most of these things will be on 3 nm by the end of this year with the Rubin launch and all the chips in that family, the GPU being the most important one. Yet Nvidia is getting the majority of supply. Part of this is because you look at the market and TSMC and others forecast market demand in many ways, but it’s also the market signal. The market signaled, “Hey, we need this much capacity next year. We need this much. We’ll sign non-cancelable, non-returnable. We may even pay deposits.” Nvidia just did it way earlier than Google or Amazon. In some cases, Google and Amazon had stumbling blocks. One of the chips got delayed slightly by a couple quarters. Trainium and all these sorts of things happened. In that case, there was a huge sort of, “Well, these guys are delaying, but Nvidia is wanting more, more, more, more. And we are checking with the rest of the supply chain, is there enough capacity?” They’re going to all the PCB vendors and saying, “Is there enough PCB?” Victory Giant is one of the largest suppliers of PCBs to Nvidia, and they’re a Chinese company. All the PCBs come from China, or many of them. They’re like, “Do you have enough PCB capacity? Great. Hey memory vendors, who has all the memory capacity?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.