Evidence receipt / evaluation
Published · transcript-backedArchie Abrams: evaluation
7 Nov 2024 Lenny's Podcast Breaking the rules of growth: Why Shopify bans KPIs, optimizes for churn, prioritizes intuition, and builds toward a 100-year vision | Archie Abrams (VP Product, Head of Growth at Shopify)
“Uh oh, we're going to make suboptimal investment decisions now by kind of moving things around even though it's the right thing to do. So a lot of it's in rebuilding all instrumentation, how we do LTV modeling, how we do attribution, how we do incrementality testing across each of those different types of outcomes, because it was not an intuitive thing for us originally because we had built all of these systems with a much more siloed view.”
Source trail
Everything needed to verify it.
- Speaker
- Archie Abrams
- Attribution
- Verified speaker
- Claim type
- evaluation
- Recorded
- 7 Nov 2024
- Publisher
- Lenny's Podcast
Transcript context
…Is there an example of something that broke that would be illustrative of what you're describing? Yeah. So I think that breaks is drive someone from an ad over to self-service. We typically look at only the self-service LTV of that person. But what happens if they come in, they sign up via self-service and then they go talk to sales? They get changed to a sales driven merchant, which means that that value of that merchant, which is usually actually quite large, does not get associated back to that ad campaign. Oh, guess what that means? That means you would probably reduce investment on that ad campaign because you weren't valuing that. Our system had two different models for calculating LTV. Sales driven one, and a self-service one. Uh oh, we're going to make suboptimal investment decisions now by kind of moving things around even though it's the right thing to do. So a lot of it's in rebuilding all instrumentation, how we do LTV modeling, how we do attribution, how we do incrementality testing across each of those different types of outcomes, because it was not an intuitive thing for us originally because we had built all of these systems with a much more siloed view. Yeah, basically attribution gets a lot more complicated. Are you going in a multi-touch attribution direction or is there something even more clever?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.