Ben Gilbert: belief

Transcript context

…You mentioned energy here. This is also Jensen’s argument. He’s like, yes, these things take a ton of energy, but the alternative takes even more energy. We are actually saving energy if you assume this stuff is going to happen. Now, there’s a bit of caveat here in that it can’t happen except on these types of machines. He enabled this whole thing, but he has a point. I totally buy it, though. I think there’s a very real case around, look, you only have to train a model once and then you can do inference on it over and over and over again. The analogy I think makes a lot of sense for model training is to think about it as a form of compression. LLMs are turning the entire internet of text into a much smaller set of model weights. This has the benefit of storing a huge amount of usefulness in a small footprint, but also enabling a very inexpensive amount of compute—again, relatively speaking—in the inference step for every time that you need to prompt that model for an answer. Of course, the trade-off you’re making there is once you encode all of the training data into the model, it is very expensive to redo it, so you better do it right the first time or figure out little ways to modify it later, which a lot of ML researchers are working on. I always think a reasonable comparison here is to compress a zillion layer Photoshop file. For anybody that’s ever dealt with a three gigabyte Photoshop file, that’s not a thing you’re going to send to a client. You’re going to compress it into a JPEG and you’re going to send that. The JPEG is, in many ways, more useful as a compressed facsimile of the original layers comprising the Photoshop file. But the trade-off is you can never get from that compressed little JPEG back to the original thing. I think the analogy here is you’re saving everyone from needing to make the full PSD every time because you can just use the JPEG the vast, vast majority of the time. Hopefully we’ve now painted a relatively coherent picture of both the advances that made the generative AI opportunity possible, that it has truly become a real opportunity, and why Nvidia even above the obvious reasons, was just so well-positioned here, particularly because of the data center–centric nature of these workloads, and that they had been working so hard for the past five years to fundamentally re-architect the data center. On top of all this, Nvidia recently announced yet another pretty incredible piece of their cloud strategy here. Today, like we’ve been saying, if you want to use H100s and A100s—say you’re an AI startup—the way you’re probably going to do that is you’re going to go to a cloud, either a hyperscaler or a dedicated GPU cloud like Crusoe or CoreWeave or Lambda Labs and the like, and you’re going to rent your GPUs. Ben, you did some research on this, so what does that cost?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.