Robby Stein: evaluation

Transcript context

…I love this topic. It makes me think about just this balance. People always try to find between optimizing something they've already got versus trying to take a big bet on something. You've had so many examples where you've taken a big bet on something totally new and it's worked out incredibly well. Do you have just a heuristic in how you structure teams and prioritize across, okay, we have amazing Google experience today, what percentage of resources go into improving that versus trying something totally new? That's one where I actually do feel like the more analytical, systematic thinking helps a lot because you're trying to produce value in the world, you want to quantify it some way. And so if you're seeing this growth curve and you're trying to understand, wow, people are using it more and more to liken this product. And when products are young, they grow, and then eventually things mature. You can break out product suites and different features of products all along the same way. Certain features that are growing fast, other features that are not. You get to these points of just diminishing marginal return in every system where it feels like you could put 50 people on this project, it's just not going to dramatically move the needle. Part of it is this bottoms up thing with your own team being really thoughtful about what is the expected value of that investment, and knowing when it's starting to approach zero or diminishing marginal return. And then when that happens, these are these moments that usually coincide with something fundamental changing. Either people's expectations, externally, market saturation, there's something happening where you need to adjust. You then find your next growth driver or set of drivers. That's where you need to go more first principled and try these new things more. Then when you land a new thing that creates this new little growth engine and then you put people on it and you optimize it because each change is like 10% win, 20% win, 4% win. It's clearly still has so much value in headroom and to make it better for people, and you can see that in the data. And so that becoming, I talked about this instrumentation, it becomes your guide for knowing if you're making good calls. Otherwise, if you don't know where you're headed and you don't have a goal of what you're trying to do more quantitatively, it's really hard to know if the thing you're doing is mattering to anyone. I think I made the product better, but is anyone using it? Does anyone care? Or are we just congratulating ourselves? Ultimately you want to have impact on people and that's what matters. So it says essentially tracking S-curves on every product and understanding if you're in the plateau and if it's time to invest heavily somewhere else.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.