Cristopher Moore: belief

Transcript context

…It's just so interesting that a layer of in the, you know, the self attention layer, it's it's this first order genocide pooling where you just kind of, you know, take all of the possible pairs of the tokens and you stack it n times, stick an MLP on the end. And like, just the sanity test there just, you know, seems to me. How could it possibly learn a deeply factorized structured representation of problems? Yeah. I I agree with you, and yet it seems to work surprisingly well at a lot of things, and we keep moving the goalposts, and we should move the goalposts. Right? The, you know, the interesting area of of research is the velocity of the goalposts. And, you know, 1 thing that I do on the side is I design puzzles. So there's this fantastic YouTube channel called cracking the cryptic, where these 2 puzzle champions from England, Mark and Simon, do, pencil puzzles, and many of them are, well, they're online nowadays. Many of these are modern variants of Sudoku. And so, you know, people take traditional Sudoku and they invent all these cool new rules. Like, there are thermometers, which are paths along which the digits have to increase, or there'll be a box within which you're told what the total of the digits in that box is, or there are additional constraints like cells and knights move apart have to be different. So there's an AI company, I think, Sakana. Oh, yes. I know them. Yeah.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.