David Rein: evaluation

Transcript context

…And software engineers, if you talk to them, they they love AI. They they they say this is a golden era. I mean, can attest to this personally. It's never been well, it's fun and stressful at the same time. It's like a slot machine. I've never been more burned out, but I'm having a lot of fun in the process. But, you know, it's just possible to build incredible things. But the narrative is that labor market disruption and having expertise in software engineering will be penalized. You know, software engineers will no longer be paid such ridiculous salaries. And and I think the complete opposite is true. I I think that this technology actually broadens the gap. So the more competence you have for software engineering, the more stuff you can get done. It's it's like a golden era and and all of this. And there's also this interesting note that you published, I think, last month on SWE bench, you know, that said that roughly half of the testing PRs from recent agents wouldn't be merged by maintainers. So, like, how do we make sense of this? So, know, on the 1 hand, you know, the the best software engineers are having a great time. On on the other hand, the the code it's producing is is is fractionated and and bad. I mean, do we understand this? Yeah. So I mean, 1 thing to say, off the bat is, whether an entire field is automated or in order for software engineering to be automated, AI systems would need to be able to do a really, really large fraction of the tasks. Basically like 100% of the tasks that are involved in software engineering. And it seems pretty clear that right now AI systems cannot do close to 100% of the tasks that software engineers do broadly. I could throw out numbers, but it's way, way lower. It might be very low or something. You know, there there is kind there there are kind of like standard results in in in economics where, you know, if you make you know, as as you if you if you automate like a small fraction of some some labor market, then it can be the case that actually, yeah, it becomes more profitable to work in that market because you're more productive, which I think is kind of how I understand what's happening now. But, you know, if it does end up being the case that 99.9% or 100% of the work of software engineering is able to be done by AIs, then I think it's, yeah, kind of hard to imagine human software engineering being relevant, or at the very least, humans would need to do very different kinds of work that maybe it's the case that there are novel tasks that current software engineers aren't doing. That once you have AIs that can do all the tasks that current software engineers are doing, now humans can switch what they're doing and make yeah, I don't know. There's this like you can imagine people being CEOs of these AI agent companies or whatever. Whether we call that software engineering or not might be a semantic thing or something. Yeah. So on the SWE bench maintainer and merge ability things, I think I was pretty curious there. Yeah. It's like, obviously, this number is gonna be lower than the okay. It is not strictly obvious. It could be that some that a bunch of the tests are unfair and actually, like, you know, the the the agents have correct solutions, but the error message doesn't match exactly or something. I think you do see this sometimes. Something like half of test passing, SWE bench solutions, wouldn't be mergeable, or or they get merged at more specifically, they get merged at about half the rate that human, golden solutions that were actually merged are merged by, you know, a different sample of of maintainers. So so there's there's was that an interesting fact. You know, if you see, like, oh, 50% of the agent solutions are rejected, it's like, well, 40% of the humans, know, human accepted solutions are rejected. So so that by itself is not but it you know, the the the rate is is half. It could be that, like, basically, the actual maintainer merge rate is, you know, pretty, flat over time and, like, you know, most of the performance increases from something like overtraining or, like, reward hacking on the benchmarks. You know, that's that's not what we saw. I'm not quite sure what is within error bars or or not. I think it it's you know, the merge ability is going up over time, and I think it's also going up as a fraction of, you know, like, conditioned on test passing, but I I think probably less confident than that. So so it's like, again, this this thing is worse, but it's not like it's it you know, it's being dragged up over time probably by, you know, the the sort of auto auto checkable thing. Yeah. And, yeah. I was also gonna say about the, yeah, like, employability as a function of automation of your job. Like, I think yeah. And people use, like, bank tellers as as an example, I I think. 1 other analogy, though, you could do do is talk about horses. Like, you know, there was a period where, like, equipment for using horses to do labor was, like, improving, and the the demand for horses increased when you have, like, you know, carts, you can use them to carry more things than just riding a horse or whatever. But then at some point, you get, like, tractors and cars, and then there is no demand for horses or, you know, bay you know, basically none. So so you can see this, like, thing where there is increasing demand as a and then once, you know, close to a 100% functions automated, it plunges. So so we could see something like that with humans. We kind of think of a lot of labor as being quite static and automatable.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.