High Signal Podcasts Evidence ledger
Method
Browse
← Back to evidence

Evidence receipt / evaluation

Published · transcript-backed

Alex Hardiman: evaluation

13 Nov 2022 Lenny's Podcast An inside look at how the New York Times builds product | Alex Hardiman (CPO at The New York Times)

“Those algorithms, what I think is just really great is they're trained on editorial signal and then they can still work towards driving towards outcomes like reach, engagement, conversion, et cetera.”

— Alex Hardiman

Source trail

Everything needed to verify it.

Speaker
Alex Hardiman
Attribution
Verified speaker
Claim type
evaluation
Recorded
13 Nov 2022
Publisher
Lenny's Podcast

Transcript context

…Absolutely. I'm going to just focus on two themes that I think are pretty interesting. The first is just how we work at the New York Times, and we talked a little bit about working with journalism, and there are some really interesting differences. The second is just on the idea of impact. I think how the definition and the understanding of impact can be pretty different. First, just on the idea of how we work, there are a lot of similarities. I would say that product managers at the Times and at tech companies, they have a lot of the same skills. Like, we look for a great product sense, great execution, great leadership and drive. Any good PM needs to know their industry, their customers, their market, their business, et cetera. We actually do see a lot of crossover between product managers from tech companies who come to the New York Times or tech PMs at the Times who go over to tech companies. I think that that's wonderful, but a key difference of when you're a product manager working at the New York Times is that you work across the full stack of the product. Meaning, we own our journalism and our content. We own our distribution and we own our products. That's really different from working at a big tech platform. When I was at Facebook, we controlled the software and the distribution, but we didn't control the content. We had real limitations on understanding what was passing through. Was it high quality content, low quality content? It just led to, again, a lot of challenges that we already talked about. At the Times, when I think about how our best products are born, it's when you bring journalism and product lovers together. That means that PMs at the Times really need to understand the blend of art and science. They really need to value expert editorial judgment as they're also looking at individual KPIs, customer research and insights, et cetera. An interesting example, as I was trying to think about what would feel really different doing product at the Times compared to say Facebook is like, let's say you're at the product team and you're working on the home screen. We always start with expert editorial judgment to curate the most important and interesting stories. But on top of that, we're training algorithms on specific data sets, like editorial important scores that actually come from our journalists. What that allows us to do is actually scale editorial judgment to a large group of readers. Those algorithms, what I think is just really great is they're trained on editorial signal and then they can still work towards driving towards outcomes like reach, engagement, conversion, et cetera. That's just such a different way of thinking. When I was at Facebook and we were focused on news ranking and feed, all we could do was train pieces of in information based on an engagement outcome. We couldn't actually train it based on the quality of that piece of information itself. sed on news ranking and feed, all we could do was train pieces of in information based on an engagement outcome. We couldn't actually train it based on the quality of that piece of information itself. At the Times you get all, you have 2000 plus journalists and you're actually trying to structure their expertise into things that can actually translate into really great algorithmic decisioning, and that's just so different. No one else is really doing something in that space. Product managers are becoming very editorially minded and we're also getting editors to become more product minded. I just think that how we work there is so different and so unique. It's just a pretty fascinating part of, I think, how the sausage is made, if that makes sense.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.

Search evidence