Dylan Patel: evaluation

Transcript context

…Yep. What it is, is there’s all this experimentation you do at the small scale, research ablations. You have your Jupyter Notebook where you’re experimenting with MLA on three GPUs or whatever and you’re doing all these different things like, “Hey, do I do four active experts, 128 experts? Do I arrange the experts this way?” All these different model architecture things, you’re testing at a very small scale. Right? A couple of researchers, few GPUs, tens of GPUs, hundreds of GPUs, whatever it is. And then all of a sudden you’re like, “Okay, guys. No more fucking around. No more screwing around. Everyone, take all the resources we have. Let’s pick what we think will work and just go for it. YOLO.” And this is where that sort of stress comes in is like, “Well, I know it works here, but some things that work here don’t work here. And some things that work here don’t work down here in this terms of scale.” So it’s really truly a YOLO run. And there’s this discussion of certain researchers just have this methodical nature. They can find the whole search space and figure out all the ablations of different research and really see what is best. And there’s certain researchers who just have that innate gut instinct of like, “This is the YOLO run. I’m looking at the data. I think this is it.” This is why you want to work in post-training because the GPU cost for training is lower. So you can make a higher percentage of your training runs YOLO runs.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.