Logan Kilpatrick: belief

Transcript context

…If you take it one step further, Folks are doing more and more of these like model welfare checks and interviews where they just literally ask the model in some cases like, how do you feel about the way that you are deployed? Is anything like that happening within DeepMind? That's a good question. I think the how is it being deployed question. I feel like the model is just. this is my sort of personal sense of a lot of these tests. Like it's like completely out of the distribute. Like the model has no idea how it's being deployed. So it's just like pontificating in a lot of these cases. Like it's not like in the context window of any large major LLMs is like, here's the details of how you're being trained and here's sort of your serving set up and here are the people who are working on it though maybe like these are interesting things to experiment with in the future. So I think a lot of it is just like pulling out of random distribution of like the large scale, training that happens on the models. And I feel like it's actually less representative of like how the model, like it just, it's just doesn't have the context. Yeah, one reason that's true, which I was just noticing in the AI studio is I think all the models that are publicly launched at least so far still have a January 2025 knowledge cutoff. And it's honestly like amazing that they do as well in search and that they can have like, I ran a deep research on like what's, give me everything that Google has launched in the AI space and like what's even the speculation about what they're going to launch at IO. And it did like a very impressive job.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.