Phil Trammell: evaluation

Transcript context

…We were talking a second ago about why there isn’t more automation as a result of LLMs. One plausible mechanism could be, as you were saying with the O-ring theory… O-ring theory refers to the fact that the Challenger shuttle blew up because one component malfunctioned, and it destroyed the whole thing. Maybe that’s a more general model of how goods are produced in the economy. You have to make sure everything is reliable and works well. So you can’t automate an entire job to an AI right now. Even though it might be able to perform it at some probability, you need extreme reliability in order for it to not destroy the finished good. This might explain why there’s a lot less automation now than there otherwise could be. But I think it works in the other direction once AIs get advanced enough. Integrating humans into the production flow of future goods will become difficult. Even beyond the arguments about how humans will be more expensive or less capable, there will be whole production flows organized for AI labor. They’re talking in neuralese. They’re thinking many thousands of times faster. So even if there’s some comparative advantage where it makes sense to hire a human, there will be transaction costs and worries of reliability that will actually make it hard to integrate humans into future production flows. That seems right to me. In particular, I just want to distinguish between the point that if you automate nine-tenths of a job, people might shift over to the last tenth, but there might be ten times more work demanded of them. Compare that to the model of O-ring automation from Gans and Goldfarb recently. If you can only automate nine-tenths of the job, but you do it to a lower standard of quality than the human could, you might not want to automate even those nine-tenths. That’s the thing that could totally port over. Symmetrically, it could be a reason why we don’t use a human for one-tenth of the job anymore, because a human just can’t perform it to the level of quality that the AI can perform the other parts of the job, or the level of speed. They end up pulling down the quality or speed of the finished product. By the way, the model you’re talking about seems extremely plausible to me for why more lawyers, accountants, or even software engineers are not automated. There are cases where there’s a pretty good probability that the thing worked as you expect, but the thing you’re paying the lawyer for is: “No, really, my company’s not going to go under because—”…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.