Lenny Rachitsky: belief

Transcript context

…We have never chatted real time, so everybody's getting the first conversation and I'm really looking forward to it. I know that you have this pretty spicy contrarian take on experimentation and that I think you believe people run experiments too often and maybe not everything should be an experiment. Can you talk about that? Yes. So, experimentation is a really big part, growth culture, growth hacking culture, PLG culture, growth marketing culture, any kind of growth, at least in my sphere, and sometimes teams can be really, really dependent on experimentation when trying to grow a business. A great example of this would be consumer growth at scale. Think your classic big social company. I worked at one of them. I am a former employee of Meta back when it was called Facebook. When you're at scale trying to grow a social app, experimenting can feel second nature. It can feel like a necessary step in your product development process that you want to drive more signups or you want to drive better activation with new customers. So, you change some buttons, change a design, and A/B test it, and then see if numbers go up or down, and then make your ship decision based on it. I find that there's generally two reasons why a grow team wants to experiment, and one of them is to understand more precisely the metric impact of what they're building and what they're putting in front of customers, and the other is risk mitigation where you're making so many big dramatic changes that there's some risk that while this could be really great for the business, it could also be really bad and it would be good to understand that before everybody's experiencing that in production. So, with that framework in mind, sometimes you don't need to experiment. Sometimes if the business, let's say activation, right, activation rates go up 6% versus 7%, that precision actually doesn't help all that much beyond being able to say in your performance review, "Hey, I increased activation by 7%." So, it also is expensive to have folks on the ground, be it engineers, analysts, product managers, spending time understanding the results of an experiment that could otherwise be spent on road mapping, on foundational analysis, on shipping things. So, experiments can be expensive. So, with all that said, generally my advice is to experiment when you need to and to primarily see it as a risk mitigation tactic when you're making dramatic changes and to let the product development process do more work. So, spend more time with customers, be more rigorous in understanding precisely what problem you're solving, get mocks in front of people and see how they react, and hopefully have more conviction than you otherwise would when you ship something that it's okay if every customer sees it tomorrow and that the experiment doesn't actually matter as much.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.