Gwern: evaluation

Transcript context

…Now that you know that AGI is a thing that's coming, what’s your thinking around how you see your role in this timeline? How are you thinking about how to spend these next few years? I have been thinking about that quite a lot. What do I want to do? What would be useful to do? I'm doing things now because I want to do them, regardless of whether it will be possible for an AI to do them in like 3 years. I do something because I want to. Because I like it, I find it funny or whatever. Or I think carefully about doing just the human part of it, like laying out a proposal for something. If you take seriously the idea of getting AGI in a few years, you don't necessarily have to implement stuff and do it yourself. You can sketch out clearly what you want, and why it would be good and how to do it. And then just wait for the better AGI to come along and actually do it then. Unless there's some really compelling reason to do it right now and pay that cost of your scarce time. But otherwise, I’m trying to write more about what is not recorded. Things like preferences and desires and evaluations and judgments. Things that an AI could not replace even in principle. The way I like to put it is that “the AI cannot eat ice cream for you”. It cannot decide for you which kind of ice cream you like. Only you can do that. And if anything else did, it would be worthless, because it's not your particular preference. That's kind of the rubric. Is this something I want to do regardless of any future AI, because I enjoy it? Or is this something where I'm doing only the human part of it and the AGI can later on do it? Or is this writing down something that is unwritten and thus helping the future AI versions of me? So if it doesn't fall under those 3, I have been trying to not do it. If you look at it that way, many of the projects that people do now have basically no lasting value. They’re doing things that they don't enjoy, which record nothing ephemeral of value that could not be inferred or generated later on. They are, at best, getting 2 or 3 years of utility out of it before it could have been done by an AI system. Wait, your timeline for when an AI could write a Gwern-quality essay is two to three years?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.