Evidence receipt / belief
Published · transcript-backedLenny Rachitsky: belief
15 Dec 2024 Lenny's Podcast Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team)
“You come up with a hypothesis, and a strategy of here's how we think we can do this, and how we can move this, and then you just try a bunch of stuff.”
Source trail
Everything needed to verify it.
- Speaker
- Lenny Rachitsky
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 15 Dec 2024
- Publisher
- Lenny's Podcast
Transcript context
…Another thing that I'll just say is, we really resist the urge to do the big V1. And I think this is, I shared the streak goal example, where, a lot of times when we're exploring something we will say, okay, well, that's cool, how do we strip away a bunch of stuff and figure out what our core hypothesis is? And then, just ship that thing first as a V1. Because it's easy, and I've found this time and time again, it's easy to add things, features that make them win, I've worked in retention engagement long enough, I can add, I know enough things to pull, and bells to add, and whistles to make something win, but there's a lot of times where it's like cool, t [inaudible 01:11:27] because all the whistles you added, not because of what your core hypothesis was, and a lot of times if you can just really simplify what the feature is, it's also much easier to ship, it's easier to design. You're not designing for a whole system, you're designing for something much simpler. And so, getting everybody to think that way, allows us to end up shipping faster, shipping simpler, designing faster, getting faster approval, getting insight, and then doing what I talked about with streak goal, being able to run iteration after iteration after iteration, add these things iteratively. And then not only by doing this are you able to move faster, but you get confidence at each step of the way, that, hey, my series of hypotheses is actually born out. Or if it's not, cool, then we're going to drop that part of the feature, and then just ship what actually matters. If I can try to summarize the broad lesson so far that I'm hearing, and maybe you would've shared this, but I'm just thinking, if I were to try to design a company to operate, that we all operate, you essentially map all the levers that drive the business. So, you have this mapping of all the metrics that drive up to growth and daily active users. CURR ended up being the biggest specific metric to drive growth long term, so there's imagining a tree of all the opportunities you could work on, you found this is what is most connected to our growth. You basically just start mining, I don't know if mining is the right metaphor, but just looking for things that move that specific metric. You just look and poke and explore, and then once you find one, you just go real deep on trying a lot of different... You come up with a hypothesis, and a strategy of here's how we think we can do this, and how we can move this, and then you just try a bunch of stuff. There's also this element of the Arrested Development quote, "There's always money in the banana stand," comes to mind, where it's just keep working on, see there's more, there's going to be more opportunity at this. When I joined Duolingo, the PM that I took over for, Anton, who used to lead the retention team, I remember saying, "Dude, the streak, it just counts up, you guys have been testing on it for years, how much more work can we do on the streak?" And he was like, Jackson, you child... He didn't say exactly this, but this is how I felt it. Like, Jackson, you child, we're not even 30% of the way optimized. And four years later, I say that with such conviction, we are so far away from... We've made a ton of strides, but we are still so far away, and every quarter where we ship a ton of wins and improvements to the streak, it just continues to prove to me that there is so much more to be done. So, I think your framing of it is... And I would say there's a lot of thought that goes into, again, I talk about the strength of the hypothesis that you have to have as you start to build out larger feature strategy, I do think it's really important to not just do a bunch of random stuff but do it with intent, with a goal in mind, otherwise, you do end up in these local maximas. But yeah, there's still a bunch of stuff that we haven't tried that I think we have high confidence in working out and so we'll keep doing that.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.