Jackson Shuttleworth: preference

Transcript context

…I love this. Okay, so you said 10,000 DAUs, I think that references you guys measure incremental impact and absolute numbers of new daily users you're going to drive attributed to that experiment, is what it sounds like. Yeah, and we do both. We'll also look at, a lot of times I'll look at, for retention, day one versus day seven versus day 14, a lot of what I'm looking for is for us to have a better day 14 impact than better day one impact because it means that users are retaining better over time. This is particularly for users that would see a feature multiple times. I just like absolute DAU numbers because as long as you're controlling for different biases, like a recency bias or a novelty bias, it's a really easy way to just have an absolute comparison. You start to look at percentage changes and then it's influenced by who your treatment, how many users saw the experiment, but at least an absolute number is easier, in my mind, to start comparing.. Again, there are pitfalls with it, but we find that that's a pretty useful way. That lesson comes up a lot on this podcast, and that approach to experiment.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.