belief · 17 Oct 2025 · 1:04:07

Labs are optimizing their flops and cost budgets by reducing pre-training scale and increasing investment in post-training stages like reinforcement learning.

The labs are just being practical. They have a flops budget and a cost budget. It just turns out that pre-training is not where you want to put most of your flops or your cost. That's why the models have gotten smaller.

Watch at 1:04:07