A scalable objective function is essential for training advanced AI models, with pre-training and RL objectives serving as examples.
The fifth is that you need an objective function that can scale to the moon. The pre-training objective function is one such objective function. Another is the RL objective function that says you have a goal, you're going to go out and reach the goal. Within that, there's objective rewards like you see in math and coding, and there's more subjective rewards like you see in RLHF or higher-order versions of that.