Dwarkesh Patel: belief

Transcript context

…I don’t know if it’s 100x. It’s probably at least 10x. Some people think ideas haven’t gotten much harder to find, so why would we need this 10x increase in research effort? To me, this is a very natural story. Why is it natural? It’s a straight line on a log-log plot. It’s a deep learning researcher’s dream. What is this log-log plot? On the x-axis you have log cumulative research effort. On the y-axis you have log GDP, OOMs of algorithmic progress, log transistors per square inch, log price for a gigawatt of solar energy. It’s extremely natural for that to be a straight line. It’s classic. Initially, things are easy, but you need logarithmic increments of cumulative research effort to find the next big thing. This is a natural story. One objection people make is, “isn’t it suspicious that we increased research effort 10x and ideas also got 10x harder to find, perfectly equilibrating?” I say it’s just equilibrium—it’s in a endogenous equilibrium. Isn’t it a coincidence that supply equals demand and the market clears? It’s the same here. The difficulty of finding new ideas depends on how much progress has been made. The overall growth rate is a function of how much ideas have gotten harder to find in ratio to how much research effort has increased. This story is fairly natural, and you see it not just economy-wide but also in the experience curve for various technologies. It’s plausible that institutions have worsened by some factor. Obviously, there’s some sort of exponent of diminishing returns on adding more people. Serial time is better than just parallelizing. Still, clearly inputs clearly matter. I agree, but if the coefficient, of how fast they diminish as you grow the input, is high enough, then in the abstract the fact that inputs matter isn’t that relevant. We’re talking at a very high level, but let’s take it down to the concrete. OpenAI has a staff of at most a few hundred directly involved in algorithmic progress for future models. Let’s say you could really arbitrarily scale this number for faster algorithmic progress and better AI. It’s not clear why OpenAI doesn’t just go hire every person with a 150 IQ, of which there are hundreds of thousands in the world. My story is that there are transaction costs to managing all these people. They don’t just go away if you have a bunch of AIs. These tasks aren’t easy to parallelize. I’m not sure how you would explain the fact that OpenAI doesn’t go on a recruiting binge of every genius in the world? Let’s talk about the OpenAI example and the automated AI researchers. Look at the inflation of AI researcher salaries over the last yea. It’s gone up by 4x or 5x. They’re clearly trying to recruit the best AI researchers in the world and they do find them. My response would be that almost all of these 150 IQ people wouldn’t just be good AI researchers if you hired them tomorrow. They wouldn’t be Alec Radford.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.