Archie Abrams: evaluation

Transcript context

…Awesome. Is there an example by any chance that comes to mind of something like that that's just like, "Wow, that was a big win. And then oh I see we just pulled forward some revenue from the future." So I think one good example is something around payment failure notifications. So one of the things that a lot of teams have or see is we call Dunning effects where somebody might have a payment not go through, a credit card that doesn't go through. So we did a bunch of experimentation around, hey, how can we alert people that their credit card is failed, their payment attempt failed? And that's a typical growth win, usually produces a lot of short-term impact. And that's what we saw here. We were doing much better alerting, reminding people, sending them a million emails about it. Cool, we got some pretty major lift. You look back six, 12 months. There was really no long-term lift. And why is that? Because there's really a little bit of a selection bias there that people who were letting that payment fail probably weren't actually that dedicated to this entrepreneurship craft. They may have updated their credit card, but they still really weren't in it. And so that was a good example of in a bunch of this stuff around payments, even "preventing" churn where you look, it's six, 12, 18 months. On a GMV metric, not a lot of lift over that long-term horizon. I love this example. I could see so many people having run experiments like this and like, "Oh, we found such a huge win. This team's killing it. What a great idea. Of course this makes sense," and then turns out it's nothing long term.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.