LLMs predict tokens (subword units) and sample from a probability distribution to generate text autoregressively.
It can never predict the next word exactly. And so what it's gonna do is produce a probability distribution of all the possible words in the dictionary. In fact, it doesn't predict words, it predicts tokens that are kind of subword units. And so it's easy to handle the uncertainty in the prediction there because there's only a finite number of possible words in the dictionary, and you can just compute a distribution over them. Then what the system does is that it picks a word from that distribution. Of course, there's a higher chance of picking words that have a higher probability within that distribution. So you sample from that distribution to actually produce a word, and then you shift that word into the input. And so that allows the system now to predict the second word, right? And once you do this, you shift it into the input, et cetera. That's called autoregressive prediction, which is why those LLMs should be called autoregressive LLMs, but we just call them at LLMs.