High Signal Podcasts Evidence ledger
Method
Browse
← Back to evidence

Evidence receipt / belief

Published · transcript-backed

Jackson Shuttleworth: belief

15 Dec 2024 Lenny's Podcast Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team)

“Don't get super caught up in it has to be like this, just test a bunch of stuff and see what speaks most to users. Because I think, again, you will constantly be surprised by the insights that you get from whether you...”

— Jackson Shuttleworth

Source trail

Everything needed to verify it.

Speaker
Jackson Shuttleworth
Attribution
Verified speaker
Claim type
belief
Recorded
15 Dec 2024
Publisher
Lenny's Podcast

Transcript context

…600, wow. ... we're running an experiment. And they range from big, like this, changing how the mechanic works, to let's swap a string with another string and see if that copy is better for users. So, we're constantly testing on everything. I do think that, I'd be more careful running that experiment now, at some point your streak gets big enough that, again, I got 9 million users on the streak, I got to be really careful... Those are our best retaining users, you got to be careful. But in the early days of the streak, I'd say test everything. See what... Don't get super caught up in it has to be like this, just test a bunch of stuff and see what speaks most to users. Because I think, again, you will constantly be surprised by the insights that you get from whether you... We shut down about half of our experiments, so half of our experiments lose, we still learned a ton by virtue of running them. So, super, super valuable. That's actually a good success rate, a lot of companies have only 20% of experiments be positive. What's your policy on if it's neutral, do you ship it, or do you kill it?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.

Search evidence