High Signal Podcasts Evidence ledger
Method
Browse
← Back to evidence

Evidence receipt / commitment

Published · transcript-backed

Jackson Shuttleworth: commitment

15 Dec 2024 Lenny's Podcast Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team)

“Everything that I'm talking about now was just that screen that day, and then was all thrown out. So, that optionality was a huge insight, and so because of that, we built a goal setting feature where you could choose between different goals, giving users that optionality was likewise a huge win.”

— Jackson Shuttleworth

Source trail

Everything needed to verify it.

Speaker
Jackson Shuttleworth
Attribution
Verified speaker
Claim type
commitment
Recorded
15 Dec 2024
Publisher
Lenny's Podcast

Transcript context

…Found a- Yeah, we found our thing, and now let's just beat the heck out of it. So, we followed that up with another experiment, where we tested different lengths. So, we test 14 days and 50 days, and we found that they were all good, but they appealed to different users. And so, we started to realize, all right, well, we probably need to be more thoughtful about who we give these different options to. And so, then we followed that experiment up with, all right, let's start with 30 days, and then we're going to let you opt out, we'll say, no, I don't think I can hit it. And our thinking was, and then if you say no, we'll hit you with an easier goal. Because we just wanted to get you to commit to a goal. And this was a fascinating one because it was a good win to give users that easier goal, to try to capture them before they said no, but it was almost just as big a win to add that opt out button. So, we tested separately, and I'm a huge fan of testing way too many arms for an experiment, just to be able to isolate your hypotheses, but we captured just what happens if we add an opt-out button, and adding an opt-out button... And you would think, as a PM, oh, now users might not engage with my feature, that's a bad thing, but it was a huge win to let them do that. And the learning here was that this intentionality of saying, no, I want it... Previously it was just a continue button, but now it's like, no, I want to hit 30 days, and having that be an intentional decision for them, yes or no, even though again, this had no impact, or no impact past this screen. Everything that I'm talking about now was just that screen that day, and then was all thrown out. So, that optionality was a huge insight, and so because of that, we built a goal setting feature where you could choose between different goals, giving users that optionality was likewise a huge win. I'll say one final learning on this, again, you talk about friction, and good and bad friction, we thought once we built a goal picker screen, where you could pick between different streak lengths, we were like, oh, well, let's recommend that users do a harder goal, thinking that okay, well, a harder goal is going to be better retention, and we'll the preselect harder goal for them. And based on all the learnings that I just shared with you, you probably imagine lost pretty significantly, we realized that yes, we could speed users through the screen more by virtue of picking a goal for them, but that act of selecting, I think it's 30 days, I think it's 14 days, was where we were getting so much of the engagement from this feature. There's so much human psychology that you all learn through all these experiments of just how to motivate people, what motivates, what demotivates, it feels like you guys should write a book on human psychology and motivation.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.

Search evidence