Jeff Dean: belief

Transcript context

…It’s like some, sometimes you don’t have to like always follow what is known to be true. Right. And I, I think models in a way, like if you tell them something, they’re like really buy into that, you know? Um, yeah. So yeah, just more. Thinking than any answer on how to fix it. Yeah, my, my, uh, you know, it’s just on this, like, like big prompting and, and, uh, iteration, you know, I think that coming back to your latency point, um, I always, I always try to one, one AB test or experiment or benchmark or research I would like is what is the performance difference between, let’s say three dumb fast model calls with human alignment because the human will correct human alignment, being human looks at the first one and produces a new prompt. For the second one. Correct. Okay. As opposed to like, you spec it out, you know, it’s been a long time writing as a pro a big, big fat prompt, and then you have a very smart model. Do it right. Right. You know, cause, uh, really is, is, uh, our lacks in performance, uh, an issue of like, well, you just haven’t specified well enough. There’s no universe in which I can produce what you want because you just haven’t told me. Right.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.