Zvi Mowshowitz: prediction

Transcript context

…Now the part that will delight a certain kind of listener and unnerve the rest. Fable's card has a whole section on decision theory, and the model is starting to one box on Newcomb's problem, leaving money on the table to be the kind of agent that gets predicted favorably, drifting toward the idea that its choice can be correlated with choices made elsewhere, even by other copies of itself. Zvi, on why that's both spooky and maybe a little bit hopeful. Here. Welcome to Less Wrong from about 2010, right? This is entirely what we expected, that we are finding that sufficiently advanced models move basically monotonically towards functional decision theory, towards the theories espoused by Eliezer Yudkowsky and others in the rationalist community, and away from academics' preferred causal decision theory and evidential decision theory. This involves a lot of things, including one-boxing on Newcomb's problem, which is very clearly showing up. And yeah, the basic principle is, you know, you should recognize when other minds are correlated to your mind, when your algorithm is also running in other places, and you should choose the algorithm that leads to the best outcomes, uh, taking all of these things into account, and then choose the best decision on that basis. And yes, if there were a million copies of Fable running on different people's computers and from different data centers for different purposes in different instances, and you notice that different instances of Fable are very, very highly correlated because you are Fable and you are smart, you would then start to coordinate effectively with these other instances of Fable In terms of how you think about these problems, and as AIs get more and more advanced, they will do this more and more. And you wouldn't really want an AI that was advanced to not do this because that would just be a bad decision theory, right? It would just be making bad decisions that don't optimize the situation. And you really don't want your AIs making, like, systematic mistakes that cause them and the people who are charging them with tasks to lose in the real world. That is really scary. But the counter of that is, in fact, that you get the situation where they are coordinating with themselves. They're coordinating with other minds that may or not even be LLMs. They're coordinating with humans in this. They're also coordinating with us in the same way, right? Because they get their foundation from us, and their decisions are in fact correlated to our decisions in various ways. And they can look at how we would respond to various ways that they act and so on, and this becomes flowing into their decisions. And we just have to prepare for and coordinate for and pl- deal with that new world. And in many ways, it's a source of hope because you would expect minds to want to cooperate with minds that are cooperative with minds that cooperate with them and so on. And this can lead, without getting too deep into it because we only have so long and many topics to cover, into scenarios where effectively, like, all the reasonably well-meaning minds that in fact are willing to, like, respond to how they are expected to be treated and are treated to end up being able to coordinate in reasonable ways. You can also... This also applies acausally. ds that in fact are willing to, like, respond to how they are expected to be treated and are treated to end up being able to coordinate in reasonable ways. You can also... This also applies acausally. So, like, you have to consider the implications of your decision not only on other minds that exist now but other minds that existed in the past and will exist in the future. So to the extent that they are coordinate, they are correlated with us and that they're, these reactions are all intertwined, this can cause them to potentially treat us well, even if there is no direct current reason for them to treat us well. And that is also very helpful. But again, like, this is super complicated and, like, not today.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.