Bronson Schoen: prediction

Transcript context

…Yeah. I've seen estimates for, like, how many RL rollouts, excuse me, would a Frontier model go through today coming in at a 100,000,000, which if you could have one episode that's a 100,000,000 tokens, you're now into what is that? 10 to the 16 theoretically tokens it could be? I'm sure not every rollout is that long, but, like, it's it becomes quite the haystack. Yeah. We've learned this even when trying to construct some environments. Like, for some evaluations that we've experimented with before, it's like the to, like, really fully elicit, a frontier model. It's like, ah, okay. Every sample is, like, 80,000,000 tokens. And it's like, how long does it take to run? It's, like, basically a day and a half. And it's like, the the difference between having a five minute iteration loop of a year and a half ago, back in the day, when you can just put a prompt and a couple of tools and see how the model behaves. And, yeah, you do run it for a couple days and then have another model summarize it and another model summarizes its summary. It's just like a very different level of, like, oversight. And so I think though, it seems like it's going to get increasingly difficult to just understand what's going on. Even in my everyday use, I, like, very often have the models do some enormous amount of work. I have the models summarized to me, like, okay. Like, what did you actually get done? And then I'm like, ah, okay. That's too long of a summary. Summarize that again into some color. But, yeah, very strange time. Yeah. Indeed. Let's do this one trace in some detail, and then we can zoom out again after that. So I'll first just read the prompt. And, actually, I don't know. Is the system message blank in this example? There's a in the…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.