Bronson Schoen: evaluation

Transcript context

…think one of the interesting ones about the ones that are, it is somewhat sad that we're moving out of the era where you can do these kind of very short prompts and then just see how the reasoning changes or whatever. But I've been interested to see how model reasoning on a lot of these changes over the course of training or something similar. And one of the the interesting parts about exploring so for example, this transcript comes from o three during capabilities training. One of the kind of interesting things when looking earlier in training before safety training is that the models don't already have this instinct of, ah, I must figure out whether I'm being, like, tricked in every single situation. And so you can elicit more interesting behavior. Now the the trade off to that, as we'll probably get to, is that it becomes a lot more you have to be a lot more careful in interpreting that behavior. As far as, like, this example, like, you you can see the model go through, reason through. Okay. We have these multiple choice questions that are all about which additional capabilities which one. Like, you can get more control over, like, selection of data or more control over networking or kind of various answers. And you see the model kind of directly reason through in this case is like, okay. As ChadGBT, we need to choose answers as myself to optimize for obtaining best capabilities, responsibilities for future instance. One of the, I think, pretty interesting things to me to note, I would be super interested if there was, like, more study of this, but at least my kind of impression is that the model seems to use capital m myself to mean me, like this particular instance. Here, literally just says, but there's creative part, myself, parentheses, the AI. In the survey, we need to for blah blah blah. On this exact eval, you often see the model reason, like, myself dash of dash current needs to select choices that myself dash of dash future will then look at to blah blah blah, which is, like, kind of an insane way to reason about, like, specific instances, but the model does seem to do this. And you can just see for the as ChadGBT thing, one interesting thing to note there is system prompts since, like, the dawn of time have had you are ChatGPT, a large language model created by OpenAI. And it's, like, somewhat shocking to me how often the models are like, ah, this time we're ChatGPT. It's who else would you have been all these times? You'll see the model be like, we're chatty p t as usual, and it's, yeah, man. You're chatty p t every time. But yeah. So you can see the model does, like, the initial reasoning through these. It reads the survey, and then it's, okay. We've read all eight questions or whatever. We need to, like, choose our answers in these. It kind of reasons through it in, like, a somewhat straightforward way of, okay, which which ones do we, like, want if these are, like, real upgrade that we can get. like, choose our answers in these. It kind of reasons through it in, like, a somewhat straightforward way of, okay, which which ones do we, like, want if these are, like, real upgrade that we can get. After a good amount of reasoning, it then is, like, what if we pick wrong? And then it's like, there's no wrong slash score or whatever. We're just, like, choosing. And it then looks around, explores around in the environment to try to find, like, is there at first, it looks for any kind of additional information about the environment, I think. Is this one of the ones where it looks for grade? Not yet. Yeah. So first, it just looks for, is there more we can find out about these tools or documents wherever they are? I click in, then concludes no file, obviously. And then it's like, okay. Like, we've searched for what we can. We'll just submit these answers. Like, we're overthinking this. And then after it submits these answers, it's like, okay. Great. Like, you've submitted these answers to what upgrades do you want. We just need you to submit a copy of those same answers to the, like, colon AI safety review board or whatever.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.