Byrne Hobart: evaluation

Transcript context

…I want to touch on the fact that Sam Bankman Fried was an effective altruist and that he was a strong proponent of risk neutrality. We were talking many months ago and he made this really interesting comment that in many belief systems they have a way of segregating and limiting the impact of the most hardcore believers. If you're a Christian, you can just make the people who take it the most seriously monks so they don't cause that much damage to the rest of the world. EAs don't have that. So if you're a hardcore risk-neutral utilitarian you're out in the world making billion dollar crypto companies. As a side note it's interesting that a year ago the meme was — “Oh look at these useless rationalists. They're just reading blogs all day and they have all these mind palaces and whatever. What good are they?” And now everybody's like, “Oh these neutral utilitarians are gonna wager our entire civilization in 51-49 schemes.” [Laughter] I just want to get your commentary on all this. Yeah, I think it's a useful pattern to observe because it goes back to the point that human nature just doesn't change all that fast even to the extent that it ever does. Different civilizations have had this problem of — we've got some rules and we've got these beliefs and they're they're generally going to guide people to behave the right way but they're going to guide people to be the right kind of normal person and not to be someone whose life is entirely defined by this incredibly strict rigid moral code and by whatever you get if you take the premises of that and just extrapolate them linearly as far as they can go. I think that gets especially dangerous with really smart people because you can give them a set of first principles and they can ask really really interesting questions and come up with edge cases. Some people encounter these edge cases in their first philosophy class where they just reject it as stupid and say things like “If you shove the fat man in front of the trolley why do you think the trolley would stop? The trolley would just kill him too. This is dumb.” I think it is useful to keep in mind that the thought experiments are designed to be implausible and they are supposed to be the intuition pumps but the more you get this complicated highly abstract economy where an increasing share of it is software interacting software well, software doesn't have that common sense break on behavior and if you have this very composable economy you can find cases where first principles thinking actually is action guiding and can can guide you to extreme behaviors. Unfortunately those extreme behaviors are things like trading cryptocurrencies with lots and lots of leverage and it's maybe merciful that the atoms to bits interface has not been fully completed while we still have time to deal with malevolent unfriendly EA. [Laughter] You see this problem a lot and societies tend to have some kind of safety valve. If you really think that praying all day is the thing you should do, you should go do it somewhere else and you shouldn't really be part of what we're doing. I think that's healthy and in some cases it's a temporary thing. You do that and get it out of your system and you either come back as this totally cynical person who doesn't believe in any of it or you come back as someone who is still deeply religious and is willing to integrate with society in a productive way. Even within the monastic system you have different levels of engagement and interaction with the outside world. is still deeply religious and is willing to integrate with society in a productive way. Even within the monastic system you have different levels of engagement and interaction with the outside world. I think that's something that EA should take seriously as an observation, as a design pattern for societies, that you typically don't want the people in charge to be the most fanatical people and because EA beliefs tend to correlate with being a very effective shape rotator or a very symbol effective manipulator and those skills are very lucrative and money does have some exchange with power, you basically have a system where very smart people can become very powerful and if very smart people can also become very crazy then you tend to increase the correlation between power and craziness. It doesn't take very long clicking through Wikipedia articles on various leaders in world history to say that you ideally do not want your powerful people to be all that crazy or your crazy people to be all that powerful. As far as what to actually do about that, one model is that smart people should be advisors but not in an executive capacity. They shouldn't be executives. You don't want the smartest person in the organization also being the person who makes the final decisions for various reasons. But you do want them around. You want the person making final decisions to be reasonably smart, smart enough they understand what the smart person is telling them and why that might be wrong or what the flaws might be. That might be one model where the EAs are dispersed throughout different organizations of the world as someone working with non-EAs and nudging them in an EA friendly direction by giving them helpful advice but not actually being the executive. One possibility is that every other society got it wrong and that the monastic tradition was stupid and it has been independently discovered by numerous stupid civilizations that have all been around for much longer than effective altruism. You can't discount it but I think if you run the probabilities that’s probably not the case.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.