Sherwin Wu: evaluation

Transcript context

…Another issue people run into, you talked about how people are shipping PRs like crazy, a lot more PRs if they're working with AI. Obviously code review is becoming a bigger challenge. Is there anything you've figured out in your team to help speed that up to make that scale and not just create this terrible job for people where they're just sitting there reviewing PRs all day? Yeah. I mean, one thing is Codex reviews 100% of all of our PRs at this point. And so I actually think, so one really interesting thing that's happened is the things that we tend to hand to the models immediately tend to be the things that annoy us or are the most boring parts of software engineering. It's also why it's more fun now because we get to do more of the fun things. For me, speaking more for myself, I really hated code reviews. It was like one of the worst things for me. And then I remember in my first job out of college, it was at Quora, I was working on the newsfeed and so I owned the code for the newsfeed. And so I was a reviewer for newsfeed and it was just like the central piece of code that everyone would touch. And so I would just, every morning I'd log in and be like 20 to 30 code reviews. I was just like, oh my goodness, I got to get through all of these. I would procrastinate and then it grows to like 50. And so there's just like a lot of code reviews. Codex is really good at reviewing code. So actually one thing that we've noticed that 5. 2 in particular has gotten extremely strongly adept at is reviewing code and especially when you kind of steer it in the right direction. And so for code reviews, yeah, we create a lot of PRs, but Codex reviews all of them and it makes code reviews go from a, I don't know, 10, 15 minute task to sometimes even just like a two to three minute task because you have a bunch of suggestions already baked in. A lot of the times people will, especially for small PRs, you actually don't even need people to review. We kind of trust Codex in this way. The original author kind of looks at Codex. The benefit of code reviews to have a second pair of eyes to make sure that you're not doing anything dumb. Codex is a pretty smart second pair of eyes at this point. And so that's something that we've heavily leaned into. The general CI process and the post kind of push and deployment process has also been heavily automated via Codex internally at this point. If you talk to a lot of engineers, the thing that annoys them the most is after you've written your beautiful code, how do you get it into production? You got to run through all these tests, you got to lint errors, code review. There's a lot of automated stuff you can do with Codex. And so we've actually built some tools internally that help automate that process, automate the lint. If there's like a lint error, it's a very easy Codex fix and it could just patch it and then kind of restart the CI process. So all of that is, we're trying to collapse into as little work for an engineer as possible, and the byproduct of which is they can now merge and push out a lot more PRs. Codex writing the code, Codex reviewing its own code. I'm curious if you are open to using other models to review your model's work. Is that a path or is it just, it's good enough, we don't need anything else.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.