Public evidence record
Published podcast speaker
Books, apps, and tools
book / recommends
“All right, I'll start with A Guide to Midwest Conversation. So, I'm based in Kansas City, I'm a proud Midwesterner, and us Midwesterners talk in a certain way. I think you hear about Minnesota nice, but we tend not to say what we mean, and it is a very funny primer into what Midwesterners actually mean when they say what they say. So, highly recommend reading that.”
Claim ledger
24 transcript-backed records
01 / recommendation
“All right, I'll start with A Guide to Midwest Conversation. So, I'm based in Kansas City, I'm a proud Midwesterner, and us Midwesterners talk in a certain way. I think you hear about Minnesota nice, but we tend not to say what we mean, and it is a very funny primer into what Midwesterners actually mean when they say what they say. So, highly recommend reading that.”
02 / uncertainty
“I don't know, sometimes it just looks a little bit nicer, it's a little more flexible.”
03 / uncertainty
“Say that you played a mobile game, or that you've done it for 3,000 days in a row, I don't know, maybe that hits a little bit different than you've learned Spanish for 3,000 days in a row.”
04 / belief
“We have very much... So, we have, if people lose their streak, there's ways to get in contact with us, but we've actually thought about that, where it's like, okay, we should just build a self-service feature, and if we think that your excuse is good enough, whatever, we'll just [inaudible 01:00:34].”
05 / uncertainty
“The amount of copy tests that we've had that have won, and I don't know, you just try things and you figure out what wins is definitely legion, and [inaudible 00:27:15] massive wins from little copy changes.”
06 / belief
“Don't get super caught up in it has to be like this, just test a bunch of stuff and see what speaks most to users. Because I think, again, you will constantly be surprised by the insights that you get from whether you...”
07 / belief
“A really process, if you're going to run this experiments, you have to be really process-oriented and really thoughtful about which experiments am I going to run when, how is that going to set up the next one, there's heavy Jira automation, I think sometimes the Atlassian suite makes my eyes bleed, but there's a lot of times where that degree of process helps the team unblock engineers and make them move really fast.”
08 / belief
“Multiple quarters of roadmap for what we can do to the streak, if other teams want to come and mess with it, okay, we got to just figure out how is that going to work with what our plans for the streak were, how do we make sure a lot of times when teams are coming in thinking, hey, let's do this to the streak, they're in context that we might have, and so there's as much of a much simpler version of what we're doing now, a bit of a knowledge sharing, of saying, all right, well, this is what we think about the streak, this is what we've seen work, hasn't worked, how does that influence some of the hypothesis that you have.”
09 / commitment
“I will say, it took a while to get people behind shouting yee-haw, but now that we have people doing it, you can't take it away [inaudible 01:26:08], we all love it.”
10 / belief
“We're planning months out, as we think about these feature iterations, even small ones, feature iterations, because when you lose cycles, not pushing on a feature, it's just sort of lost opportunity.”
11 / belief
“If you look at the numbers, I think pretty objectively has been our biggest growth lever for driving DAUs, and also say a lot of it's just related to how we think about growth at Duolingo.”
12 / commitment
“I will share the weird tradition that we do at every retention standup, and this started during the pandemic, we obviously used to stand up in person, and then when we went remote, we did this thing where whoever's the last person to go would count down 3, 2, 1, and then we'd all try to clap at the same time, which was kind of fun and dorky, but we fell in love with it, and four years later we're still doing it.”
13 / uncertainty
“I do remember trying to hire for a while a haptics contractor, like haptics design, and it was the saddest hiring I've ever done, because it was just, I don't know, it was such a specific...”
14 / belief
“I am well known for saying in the company that I think every team, every app could benefit from a streak.”
15 / evaluation
“So, used to lose a streak, we had a feature, in the day, back in the day, called Streak Repair, we'd give you your streak back, you had to pay gems. But what we found that worked way better was a feature called Earn Back, and this is basically where you would have to do a certain number of lessons, as long as you came back within a window soon after losing your streak, do a few lessons and we just give you your streak back.”
16 / preference
“Leaderboards is the best vector for doing it, so that team does a lot of leaderboards work, but every now and then I have an idea that I think will be highly retentive, and I will go in, and I'll pitch to them, and then we'll do some change to the leaderboard to make it more retentive.”
17 / preference
“We'll also look at, a lot of times I'll look at, for retention, day one versus day seven versus day 14, a lot of what I'm looking for is for us to have a better day 14 impact than better day one impact because it means that users are retaining better over time.”
18 / preference
“The unit of use for Duolingo is doing a lesson. And so, if what we care about is users coming back every day and doing a lesson, because it shows that they're actually engaging with the app, then it doesn't hurt us to make our streak focus on just do one unit versus multiple.”
19 / preference
“That is definitely a benefit, if I had an app... And that this is actually why I think a lot of mobile games do streaks differently, because to say that you played a mobile game, and as somebody who plays a lot of mobile games, that you've done it for 3000 days in a row, I don't know, maybe that hits a little bit different than you've learned Spanish for 3000 days in a row.”
20 / evaluation
“The idea of keeping a flame lit. And again, I think we've established the flame is for a lot of users as sort of their understanding of a streak, which is great, but there's a lot of people in different cultural context and different stages of life, where the idea of keeping a flame lit to show your commitment to something makes less sense.”
21 / evaluation
“We had an employee who lost her, I think a four-month perfect streak, and it was a big deal for her because she did her lesson, and she was crossing international dateline, there was a bunch of stuff going on that was like, it was just kind of weird in our backend.”
22 / evaluation
“Actually, this is my first win experiment when I joined Duolingo was, when you start a streak, we have little copy at the bottom of the screen that just, I don't know, it's like flavor copy, we use it to celebrate you, or give you context, and I ran some tests that just tried to in eight words explained what a streak was, that was it.”
23 / commitment
“Everything that I'm talking about now was just that screen that day, and then was all thrown out. So, that optionality was a huge insight, and so because of that, we built a goal setting feature where you could choose between different goals, giving users that optionality was likewise a huge win.”
24 / preference
“Typically, we have some insight because of all of these experiments that we've run, that hey, if we do this, I don't know, it's one here, or it's worked here, it's driven this user engagement, if we massage that and try in this scenario, or a different screen, we come in at least with a strong hypothesis of this will work.”