Dwarkesh Patel: evaluation

Transcript context

…I think they have “it.” The “it” factor. When I was an undergrad, I remember there was a student like this that studied with me, so I know it exists. I think it’s interesting to distinguish “it” from whatever pre-training does. One way to understand what you just said about not having to choose the data in pre-training is to say it’s actually not dissimilar to the 10,000 hours of practice. It’s just that you get that 10,000 hours of practice for free because it’s already somewhere in the pre-training distribution. But maybe you’re suggesting there’s actually not that much generalization from pre-training. There’s just so much data in pre-training, but it’s not necessarily generalizing better than RL. The main strength of pre-training is that: A, there is so much of it, and B, you don’t have to think hard about what data to put into pre-training. It’s very natural data, and it does include in it a lot of what people do: people’s thoughts and a lot of the features. It’s like the whole world as projected by people onto text, and pre-training tries to capture that using a huge amount of data. Pre-training is very difficult to reason about because it’s so hard to understand the manner in which the model relies on pre-training data. Whenever the model makes a mistake, could it be because something by chance is not as supported by the pre-training data? “Support by pre-training” is maybe a loose term. I don’t know if I can add anything more useful on this. I don’t think there is a human analog to pre-training.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.