Dwarkesh Patel: belief

Transcript context

…Yeah. Or maybe if you’re doing a lot of writing, help from LLMs and stuff like that, it’s probably bad because the models will silently give you all the same stuff. They won’t explore lots of different ways of answering a question. Maybe this diversity, not as many applications need it so the models don’t have it. But then it’s a problem at synthetic data generation time, et cetera. So we’re shooting ourselves in the foot by not allowing this entropy to maintain in the model. Possibly the labs should try harder. I think you hinted that it’s a very fundamental problem, it won’t be easy to solve. What’s your intuition for that? I don’t know if it’s super fundamental. I don’t know if I intended to say that. I do think that I haven’t done these experiments, but I do think that you could probably regularize the entropy to be higher. So you’re encouraging the model to give you more and more solutions, but you don’t want it to start deviating too much from the training data. It’s going to start making up its own language. It’s going to start using words that are extremely rare, so it’s going to drift too much from the distribution. So I think controlling the distribution is just tricky. It’s probably not trivial in that sense.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.