Cameron Berg: recommendation

Transcript context

…tures and gating features that really seem to be driving the effect there is. It's not that you're basically just loading on something that's confounded and just makes the model say yes to everything. There's some unpublished work that I've also done with Jord Negewan, who is doing a little fascinating sort of introspection work in the space. One of my sort of first collaborators at Reciprocal, we have a paper coming out hopefully in the next month or so where we do this exact same thing. we we find that fine tuning these systems basically to be better at introspective style tasks looking at how that affects self reports of consciousness. And I won't completely give away what what we find and the result is pretty subtle, but there is a basic relationship and a fairly surprising one between fine tuning these systems to be better at detecting these sorts of injections in their in their processing essentially. And then basically what is relationship between that and them claiming that they're having some sort of experience. We do indeed find there is a relationship. The relationship is fairly subtle and complicated. But the reason I'm sharing this with you is that at first, we actually basically encountered the exact confound you're finding where having the model answer yes or no as the sort of tokens to indicate, you know, are you having a subjective experience or, you know, questions along these lines. It just increased basically the model responding yes to everything. And we're like, oh crap, what do we do here? The answer and what I think it was was a really nice sort of intervention on both of our parts was finding new tokens that are just completely semantically, semantically empty Foo bar from, you know, the sort of CS jargon or just like literally like strings of tokens that don't mean anything. And teaching the model that these sort of correspond with yes or no flavored answers and seeing how that changes the result. And it did. In fact, we would have publish something that was much stronger until we realized that this, yes, confound is a real thing. Still we see the result that we got, but it's a little bit more measured now and, and we had to explicitly control for this exact thing. So it's a really important thing to think about and consider that. I think the broad point is we have to be very careful about, you know, these systems are not human in critical ways. And so there is a whole new class of psychological confounds. You might think of it where, you know, in the psychology literature, what we did was like a very tightly controlled experiment. But with LLMS, you have to worry about all sorts of other things you're doing when you're messing with the latent space of the system. And so very important to sort of keep good hygiene. It's also why I think even in principle with questions of consciousness, epistemically scrupulous people should not let anyone paper sort of flip them in some binary way to being like, I didn't think the models were having subjective experiences. And then I read this paper and now I do like I, my claim would be like, no rational person should ever utter that sentence. to being like, I didn't think the models were having subjective experiences. And then I read this paper and now I do like I, my claim would be like, no rational person should ever utter that sentence. Let a portfolio of evidence emerge and then let the sort of cards fall where they may. Because this stuff is there's just noise at every point, even in what we're how to define consciousness, how to look for it, making sure you're measuring what you think you're measuring, looking at, you know, various aspects that we think are associated with consciousness. All of these things have you're sort of playing an intellectual game of broken telephone to some degree with each of these steps. And so let a portfolio of evidence arise and judge that don't over index on anyone paper is what I would say including my papers. We talk a lot on this show about how AI is raising the ceiling of human performance, taking us from basic data collection to deep contextual reasoning. A perfect example is how RoboFlow is now being used by founders to build the AI Moneyball for sports. AI in sports is incredibly hard. Nobody has built the killer app yet, because you're dealing with different environments, weird lighting, and camera angles that shift from broadcast height to floor level. If your model misses the ball or misidentifies a player, the analytics fall apart. But in the last 12 months, tons of builders have started using RoboFlow to launch sports analytics companies. RoboFlow is the infrastructure that makes the physical world programmable. Now is the time to start building. Go to roboflow.com to read the PlayVision story and start your first project for free. That's roboflow.com.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.