Trenton Bricken: belief

Transcript context

…As we're training them on longer, and longer horizon tasks, and it takes longer for them to get any signal on whether they successfully complete the task, will that slow down progress because it takes more compute per task? I do think there's this notion that the longer, the harder tasks, the more training is required. I'm sympathetic to that naively, but we as humans are very good at practicing the hard parts of tasks, and decomposing them. I think once models get good enough at the basic stuff, they can just rehearse, or fast-forward to the more difficult parts. I mean that's definitely one of the big complexities. As you use more compute, and as you train on more, and more difficult tasks, your rate of improvement of biology for example is going to be somewhat bound by the time it takes a cell to grow in a way that your rate of improvement on math isn't, for example. So, yes, but I think for many things we'll be able to parallelize widely enough, and get enough iteration loops.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.