Guillaume Verdon: evaluation

Transcript context

…We can still think about the various ways things go wrong, for example, the founding fathers of the United States thought about human nature and that’s why there’s a discussion about the freedoms that are necessary. They really deeply deliberated about that and I think the same could possibly be done for AGI. It is true that human history shows that we tend towards centralization, or at least when we achieve centralization, a lot of bad stuff happens. When there’s a dictator, a lot of dark, bad things happen. The question is, can AGI become that dictator? Can AGI when develop, become the centralizer, because of its power? Maybe because of the alignment of humans, perhaps, the same tendencies, the same Stalin like tendencies to centralize and manage centrally the allocation of resources? And you can even see that as a compelling argument on the surface level. “Well, AGI is so much smarter, so much more efficient, so much better at allocating resources, why don’t we outsource it to the AGI?” And then eventually whatever forces that corrupt the human mind with power could do the same for AGI. It’ll just say, “Well, humans are dispensable, we’ll get rid of them.” Do the Jonathan Swift, Modest Proposal from a few centuries ago, I think the 1700s, when he satirically suggested that, I think it’s in Ireland, that the children of poor people are fed as food to the rich people and that would be a good idea, because it decreases the amount of poor people and gives extra income to the poor people. So on several accounts decreases the amount of poor people, therefore more people become rich. Of course, it misses a fundamental piece here that’s hard to put into a mathematical equation of the basic value of human life. So all of that to say, are you concerned about AGI being the very centralizer of power that you just talked about? I do think that right now there’s a bias over a centralization of AI, because of a compute density and centralization of data and how we’re training models. I think over time we’re going to run out of data to scrape over the internet, and I think that, well, actually I’m working on, increasing the compute density so that compute can be everywhere and acquire information and test hypotheses in the environment in a distributed fashion. I think that fundamentally, centralized cybernetic control, so having one intelligence that is massive that fuses many sensors and is trying to perceive the world accurately, predict it accurately, predict many, many variables and control it, enact its will upon the world, I think that’s just never been the optimum, right? Like let’s say you have a company, if you have a company, I don’t know, of 10,000 people, they all report to the CEO. Even if that CEO is an AI, I think it would struggle to fuse all of the information that is coming to it and then predict the whole system and then to enact its will. What has emerged in nature and in corporations and all sorts of systems is a notion of sort of hierarchical cybernetic control, right. In a company it would be, you have like the individual contributors, they are self-interested and they’re trying to achieve their tasks and they have a fine, in terms of time and space if you will, control loop and field of perception, right. They have their code base, let’s say you’re in a software company, they have their code base, they iterate it on it intraday, right. And then the management maybe checks in, it has a wider scope, it has, let’s say five reports, right. And then it samples each person’s update once per week, and then you can go up the chain and you have larger timescale and greater scope. And that seems to have emerged as sort of the optimal way to control systems. And really that’s what capitalism gives us, right? You have these hierarchies and you can even have like parent companies and so on. And so that is far more fault tolerant, in quantum computing, that’s my feel that came from, we have a concept of this fault tolerance in quantum air correction, right? Quantum air correction is detecting a fault that came from noise, predicting how it’s propagated through the system and then correcting it, right, so it’s a cybernetic loop. And it turns out that decoders that are hierarchical and in each level, the hierarchy are local- icting how it’s propagated through the system and then correcting it, right, so it’s a cybernetic loop. And it turns out that decoders that are hierarchical and in each level, the hierarchy are local- … that are hierarchical. And at each level, the hierarchy are local, perform the best by far, and are far more fault-tolerant. The reason is, if you have a non-local decoder, then you have one fault at this control node and the whole system crashes. Similarly to if you have one CEO that everybody reports to and that CEO goes on vacation, the whole company comes to a crawl. To me, I think that yes, we’re seeing a tendency towards centralization of AI, but I think there’s going to be a correction over time, where intelligence is going to go closer to the perception. And we’re going to break up AI into smaller subsystems that communicate with one another and form a meta system.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.