Ben Todd: evaluation

Transcript context

…This is a bit of a, I don't want to shortcut your list if there's more you want to add, but one idea that I've been coming back to over and over again is writing utopian fiction to just try to make the future the goal state a little more concrete. I think we've seen a little bit of evidence this last couple weeks that could even have a positive effect as it flows through the training data and gives AIs a conception of themselves based on our imagination of what they maybe should be like in a mature state. Any feedback on that idea writ large or if somebody was actually going to go try to seek to steer the future through utopian fiction, any advice you would give them? I heard people speculate about ideas like, can you flood the internet with positive training data and therefore influence the values of that? I don't, I presume there's a lot of reasons why that wouldn't work. With utopian fiction in particular, I do think one thing I would say is the track record is very bad. Like most people who've tried to write a utopia, when you look back on it, just seems like a dystopia to us today. And like, It's just very hard to not be very trapped in the values of your times. So I'm quite into the idea of what we should be shooting for is this idea from Will MacAskill, which he calls a Viotopia, which is not specifying a particular end state, but it's trying to just get civilization in a better position to navigate to the best end state later. So for example, one thing that would help is not having an irreversible essential risk, because then you've cut down a lot of options. But also, not, it seems not having an authoritarian government, again, would help to preserve debate and mean you have more options to navigate the future. Having more information about what's happening, these are the kind of things that seem robustly good for navigating things that a lot of people could agree on, even if we don't know the particular end state. The analogy like Will uses here is like, imagine if you're lost in the wilderness and you don't know which way to go to escape. You might, but you might still be able to say, well, I need to get water, I need to get to higher ground so I can look around. And these would still be robustly positive goals to work towards. And then like, hopefully from there later, you'd have a chance to figure out which direction you should actually walk in. Yeah, I like that. Viatopia. You've been generous with your time. I know it's getting late for you across the pond. Maybe just in closing, any other organizations that you would want to shout out that we haven't mentioned that can magically show up in the show notes? Any other topics or tidbits that we didn't touch on that you would want to make sure you leave people with? And then maybe, just tease a little bit like what else is in the book that we didn't talk about? So people are still not left with the impression that this conversation substitutes fully for the book and are still motivated to go pick it up.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.