Axel Højmark: belief

Transcript context

…And also just on terminology, so we we've been using the term reward seeking, and many folks will know about reward hacking, and you've you mentioned meta optimization, and there's a few other terms you've used in the papers. Can you can you just distinguish those things? Reward hacking is when you find unintended tricks or hacks or unintended solutions to problems that essentially developers didn't intend. So, like, a classic example is, like, in CoastRunners, I think the game is, where the the boat is optimized or, like, the RL agent is optimized to drive a boat in certain in in a in a race, and essentially, what it learns is just to drift in a corner over and over and maximize reward that way. So it's, like, 1 example of, like, very simple reward hacking. Now, reward seeking is the general is generally the situational aware reasoning about what is being graded, what is being rewarded, and then optimizing your actions for that. So reward hacking is 1 useful strategy for getting a high reward. So I think reward seeking is sort of the higher concept, and reward hacking is sort of the lower concept. Situational awareness is an important concept.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.