Tom Verrilli: evaluation

Transcript context

…So following that thread a little bit, what are some of the ways that AI has enabled you and/or your team to move faster and be more productive other than prototyping, which is the very clear benefit of AI for PMs and product teams? What else? What are maybe in the top three of, wow, this has really unlocked our productivity and the quality of what we do? I mean, the first one I think by a country mile is data science. We use Hex threads internally at Whatnot. I'm sure there are other comparable products, but I think it's almost hard to remember time as a PM before you had tooling like that where you could genuinely start pulling very nuanced cohorts of data where you could grab an individual user where you've heard a report. Actually understand, let's pull logs, help me understand exactly what this user did and saw. How many other users look like this? What would impact be? And then all of a sudden you can build pretty meaningful sensitivity models or forecasts of what might happen, regression models, et cetera, really, really, really quickly, which is incredibly powerful. The other thing that we've found is it helps us move much more quickly with shipping things because you can spot regressions and weird knock-on effects of two products into mixing more quickly than you used to be able to. And I think in really large complicated systems, that's always one of the things that ends up slowing you down of release trends and all of that versus if you build the right AI tool, you can spot regressions really quickly, which basically lets people just go. So I don't know what the future of data science looks like. But I think as a product manager, I've spent less time in the last year talking to a data scientist than I ever have in my career, even though I've probably spent 10 times more time in data and understanding actually how the product's working than I ever have in my career. So that one I think is really powerful. Second one I've already mentioned, which is stop bothering engineers with how does the code base work? And actually just go and talk to Claude and understand it, which is really helpful. I used to say early on in my career that the goal was always to be understand your systems at the boxes and lines level of which system drives which thing. And now there's no excuse not to understand that or a nuanced layer. But the other one, and this might be very specific to Whatnot, so I don't know that this will help everyone, but one of the things that I've been lucky to do in my career is basically worked on live products for a decade now. And so it's always been really cool to be able to ship a product and then watch a customer use it and watch them figure it out. So I think people have just gotten this experience with Listen Labs and others in that cohort of watching people use your product. But I've always been able to sit and watch people use a thing for the first time and go through that new user comprehension gap. What's really cool with a bunch of the AI tooling right now is as they're describing, oh, I'm having a problem, you can literally be watching the code base live and work out, is that actually a bug that's happening right now right now? Or is that a comprehension gap where it doesn't work as expected? a problem, you can literally be watching the code base live and work out, is that actually a bug that's happening right now right now? Or is that a comprehension gap where it doesn't work as expected? And suddenly you've got this video artifact of someone using your product, you can be analyzing the code base in real time, and you can be just talking through AI to the code base to understand what's actually happening. And it's like a feedback loop on steroids because all of a sudden you know exactly what's going on, customer side, code side, and observe side as a viewer in real time, which is really cool.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.