LLMs are trained by masking words in text and predicting the missing words, leading to autoregressive prediction.
the way LLMs are trained is that you take a piece of text, you remove some of the words in that text, you mask them, you replace them by black markers, and you train a gigantic neural net to predict the words that are missing. And if you build this neural net in a particular way so that it can only look at words that are to the left of the one it's trying to predict, then what you have is a system that basically is trying to predict the next word in a text, right?