Yann LeCun observes that the probability of an answer being nonsensical increases exponentially with the number of tokens produced by an LLM.
So there's a strong, like you said, assumption there that if there's a non-zero probability of making a mistake, which there appears to be, then there's going to be a kind of drift. - Yeah. And that drift is exponential. It's like errors accumulate, right? So the probability that an answer would be nonsensical increases exponentially with the number of tokens.