Ryan Greenblatt: uncertainty

Transcript context

…I just want to unpack a couple of things for the audience. The thing you’re pointing out is that the price per token has not increased that much since 2024 or 2023. GPT-4 was, I don’t know, like $30 per million output tokens? Mythos is like $50 per million output tokens. Right. So the thing you’re trying to explain is, “How can it be that we’re in this era of scaling — and so bigger models should be more expensive to serve — but the token price is not increasing?” You’re suggesting that we’ve increased active parameters slower than you would have naively assumed because people just want to make fast progress on training models. You do that by training smaller models faster.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.