Archie Abrams: belief

Transcript context

…Because where people just make a call at a certain point in time, here's the list in it, here's the list experience. I love this, because very few people have experiences running a longterm experiment, and so this is a really interesting insight that you're sharing that, I guess how often do you find this to be true in your long-term holdouts, where things end up being very different downstream? I think there's probably two things that have been very common. And I would say in quite a few cases, you get a lift on a metric up front, a more short-term metric. Number of people who become a paying shopper, number of people who make their first sale in Shopify. And then you look a year later, and there's actually no incremental lift on GMV from that cohort. And so I think it actually trains, a lot of us in growth are looking at these short-term metrics. A lot of the time it's actually more pull-forward effect, than you fully realize or an incremental user that's just really not worth that much. So that's one. And then two, so this effect size goes away. There are cases where the experiment has flipped the other way. And then there are cases, and these are the most interesting ones, where you realize that you uncovered a pocket of merchants that are actually extremely valuable entrepreneurs who go on to be successful, that you missed in your normal short-term measurement techniques. And so all across the board we see that. But actually the most common is it actually isn't a long-term lift from a lot of things that you might think of the short-term are. Is there an example in that second bucket of what you mean when you say there's a pocket of valuable merchants?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.