Cristopher Moore: evaluation

Transcript context

…Oh, yes. I know them. Yeah. Yeah. So they worked with these guys to compile a lot of these. And so the goal is can you get an AI to read these rules in English and then solve some of these sudokus? Last time I looked, the behavior so far was pitiful. It was like, you know, they had done a couple of 4 by 4 sudokus. Right? Maybe a 6 by 6. I can't remember. And, of course, these these are puzzles that are designed by humans to have interesting insights about them and, you know, cool global constraints that cause various kinds of logic which is just not present in traditional Sudoku. And so so far, the ability of AI to absorb these rules and then use that to do some kind of intelligence search, that hasn't happened yet. Now I'm sure that it will I'm sure that it will improve. I think 1 of the reasons why LLMs do poorly on these things, I think, is again this basis on of 1 dimensional text. And at least a year or 2 ago when I tried ChatGPT on very simple tasks involving 2 dimensional arrays of like, you know, like the the classic queen's problem and things like that, it it really couldn't do it. Whereas we have this sensorium. Right? We're used to being able to look at a 2 dimensional image. Our eyes, you know, our pupils can secate around very easily. 1 of the reasons why we like sudoku is it's very easy to scan a row, scan a column, and scan a little 3 by 3 box so it fits with how we can address that data structure, if you will. And, that lets us do, I think, much more, directed kinds of search. I mean, the last thing you would want to do is translate it on to a big boolean satisfiability problem, and then use your favorite boolean satisfiability solver. You could do that, but that's certainly not what Mark and Simon do on their YouTube channel. They sort of sit there and think about the rules and derive from them some heuristic or some high level logical constraint and then use that. And I don't know. For me, that's a really interesting, benchmark, and I'll be very excited and a little annoyed if AI start solving those problems. I'm proud to say 8 of my puzzles are in that dataset. And so I'm waiting to see if AI can solve my puzzles. Yes. I suppose the paradox is that even though they they are kind of compute restricted, you made a wonderful observation yesterday that, you know, we have these hard problems and the art is transforming them into simpler problems with with heuristics. So in a sense, the the intelligence is about doing more with less. It's about making hard problems simple. And if only it were possible just to to make that transformation, then the language models would be able to do it. But what kind of intelligent process do you need? I mean, what goes through your mind when you come up with these in crates you know, these creative flashes of insight?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.