Lenny Rachitsky: evaluation

Transcript context

…The only thing better than knowing... If you have data of what your customers are doing, that is the best data you could ever get. And so if you don't have a tested hypothesis, if you can't think of a way to run an experiment, then honestly that idea is pretty useless. Maybe it makes sense to the market, to the model, but you could have weird consumer sentiments. Not everyone is a rational actor. So testing the actual experience and seeing how people respond to it, that's the best possible data. Pulling that thread a little bit, for startups, experiments are often hard because there's just not enough data and enough users. How do you think startups should approach that? Can you run experiments when you're really, really early? You should. Even if you have a sample size of 30, the data you get back, generally, does not change but its precision will. So mathematically speaking, you're going to get the same level of trends, but the precision at which you understand those trends will become more deep if you have more data. But the underlying information that you're getting out of that won't be very different at larger scales. So what's better than having 30 data points? Certainly having 100. But what's better than having zero is definitely 30.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.