Jeff Dean: belief

Transcript context

…Right. There's got to be some way to get more from the same data, make it learn it forwards and backwards. And every which way. Hide some stuff this way, hide some stuff that way, make it infer from partial information. I think people have been doing this in vision models for a while. You distort the model or you hide parts of it and try to make it guess the bird from half, like that it's a bird from this upper corner of the image or the lower left corner of the image. That makes the task harder, and I feel like there's an analog for more textual or coding-related data where you want to force the model to work harder. You'll get more interesting observations from it. Yeah – oh, the what? Yeah, the image people didn't have enough labeled data so they had to invent all this stuff.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.