Lenny Rachitsky: uncertainty

Transcript context

…t thread and you just find that there is no speed limit for personal growth. In a way Shopify has been a wonderful experimental lab for this sort of conviction. I've just seen this to come to be true. And, of course, hearing from someone that you respect that "Hey, I think you had it in you to do this thing significantly better because I think you probably saw fairly early in the project this sort of path A path B. You chose path B potentially out of convenience even though you knew that wasn't the right thing. And I actually expected better of you and I expected" ... "I think the next time this happens in your career you should go path A because" ... "Based on your conviction." And, therefore, that's hard to hear, right, because it's right. But it's also extremely valuable, right? What I love is an environment of people who are holding each other accountable to the actual potential rather than sort of their current level plus or ... Plus some, I don't know, a little bit extra. There's so many threads I want to follow here. The example of the school is such a pertinent one. To me right now we're looking at preschools for our son and I'm ... They're describing their education philosophy I'm like, I don't know why I should believe this is the right approach. And it makes me just want to spend all this time researching what education approach works. I know it's just preschool and maybe not as critical yet. Did you have a bet yet? I have three kids too and this is sort of a decision that every parent faces, right? So many of your listeners are probably product managers of machine learning products that maybe this resonates. So there's a funny thing about machine learning which you're just like ... You train on a lot of data, and, hopefully, you get something that predicts the thing you want it to predict correctly out of it. The biggest problem of this is overfitting, right? What does good look like, a loss function? Which is a heuristic because it's not the actual task that the thing will do in the future it's something that proxies to the task that you want the thing to do in the future. Predict fraud, predict the next word, whatever. So overfitting is basically model learning how to cheat on the benchmark or on the fitness function. So there's a business analogy of this which is that ... It's called Goodhart's law. It's literally the same thing as overfitting just for businesses. Goodhart's law just says, "Any metric that becomes a goal ceases to be a good metric." Same exact thing. The universal truths are things that almost any competitive field will invent for itself by different terminology often. And I think this is also, by the way, why it's so interesting to focus on personal growth and learning a lot about a lot because you end up finding these sort of hidden harmonies behind things, the things that are clearly enduring correct insights. So overfitting, Goodhart's law are the same thing. School optimizes for what? Marks supposedly, right? In fact, overfitting in school is literally the kids cheating to get marks, right? You get another analogy. What is however the right loss function for children? Have you made a decision yet?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.