Beth Barnes: belief

Transcript context

…things where they're expensive to check? Maybe we should bring in, Daniel, Cockatacciolo. So, in his AI 2027 piece, he's been on the show. He's been doing the rounds. Hugely impactful piece talking about timelines. And he cites your work directly. And guess the question is, do you think in the public discourse, is this being, like, overread? Like, how do you think about the interpretation of this in in terms of of extrapolations and timelines? I mean, definitely, some people are overreading it. Like, definite you know, definitely, things are overhyped, and, you know, you see a bunch of people on Twitter saying crazy things, and people also just, like, misunderstanding even what it's measuring and and and general falling off of caveats and things. Daniel Cockatau is pretty, you know, reasonable and and sort of, you know, thinks about things in a in a probabilistic way. Think he's he's probably, like, you know, more more confident on some things where I'm I'm more uncertain, and I think some of the, like, AI futures project models are, like, more sensitive to the meteor time horizon metrics than they should be. I don't think it's crazy. Or or, you know, being like this it's it is plausible that this does capture a trend that will will transfer to other types of tasks and, like, you know, that is some, you know, story we should be thinking about, like, what if, you know, what if that's true? What happens if that's true? You know? And it's also plausible that it it doesn't, and, you know, these these things are gonna, like, diverge. You know, I I I'm producers, like, Bayesian or pragmatic or whatever. I'm like, well, we wanna make some prediction. You know, we wanna have some kind of distribution over what we think the future is gonna be like so that we can plan. So, you know, it's being like, yeah, what if this kind of trend holds, and this is roughly characterizing what will happen overall? It it seems pretty reasonable, and you should also think like, yeah. What if it doesn't? So some people are saying that software engineering is gonna be automated. And software engineers, if you talk to them, they they love AI. They they they say this is a golden era. I mean, can attest to this personally. It's never been well, it's fun and stressful at the same time. It's like a slot machine. I've never been more burned out, but I'm having a lot of fun in the process. But, you know, it's just possible to build incredible things. But the narrative is that labor market disruption and having expertise in software engineering will be penalized. You know, software engineers will no longer be paid such ridiculous salaries. And and I think the complete opposite is true. I I think that this technology actually broadens the gap. So the more competence you have for software engineering, the more stuff you can get done. It's it's like a golden era and and all of this. And there's also this interesting note that you published, I think, last month on SWE bench, you know, that said that roughly half of the testing PRs from recent agents wouldn't be merged by maintainers. So, like, how do we make sense of this? So, know, on the 1 hand, you know, the the best software engineers are having a great time. On on the other hand, the the code it's producing is is is fractionated and and bad. I mean, do we understand this? Yeah. So I mean, 1 thing to say,…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.