Sholto Douglas: evaluation

Transcript context

…Does the code pass the test? Does it even run? Does it compile? Yeah, does it compile? Does it pass the test? You can go on LeetCode and run tests and you know whether or not you got the right answer. There isn't the same kind of thing for writing a great essay. That requires... The question of taste in that regard is quite hard. We discussed the other night at dinner, the Pulitzer Prize. Which would come first, a Pulitzer Prize winning novel or a Nobel Prize or something like this? I actually think a Nobel Prize is more likely than a Pulitzer Prize-winning novel in some respects. Because a lot of the tasks required in winning a Nobel Prize—or at least strongly assisting in helping to win a Nobel Prize—have more layers of verifiability built up. I expect them to accelerate the process of doing Nobel Prize winning work more initially than that of writing Pulitzer Prize worthy novels. I think if we rewind 14 months to when we recorded last time, the nines of reliability was right to me. We didn't have Claude Code, we didn't have Deep Research. All we did was use agents in a chatbot format.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.