Grant Sanderson: observation

Transcript context

…I think it hits a little bit to your miracle year point where I think what might be happening is you have people with a ton of potential energy for something that they've kind of been thinking about making for a long time. And the hope was to give people a little push. Here's a deadline. Here's a little prize. Here's a promise that maybe if you make it, it won't just go into the void, but there's a chance that it could get exposed to more people, which I think is absolutely played out. And not for the reason that someone might expect where I choose winners and I feature those winners and people watch them. A huge amount of viewership happens before I even begin the process of looking at them. And this was an accident too, where in this first year, we got 1200 submissions. I said expect judges who are reviewing it to spend at most 10 minutes on each piece. So it could be longer, but don't rely on someone watching it for more. But realistically, when I'm reviewing something, I want to watch the whole piece. I absolutely do not have time to watch that many. I've learned it takes me about two weeks of just full time work to watch 100 of these pieces and give the kind of feedback that I want. To manage that problem of more than we could manually review, we put together this peer review system that would basically have an algorithm feed people pairs of videos. And they would just say which one is better and then it would feed them another one. And in the first two years, we just used a tool that was common for hackathons that did this. And what that did is one, it gave us a partially ordered list of content by quality loosely. We didn't need it to be perfect. We just needed there to be a very high chance that the five most deserving videos were visible somewhere in that top 100. So there the algorithm doesn't have to be perfect. A thing I've learned about the YouTube algorithm is — in theory, you would want to just use machine learning for everything. You have some massive neural network where on the input of it, it's got five billion videos or however many exist. And the output decides what seven are best to recommend to you. That is completely computationally infeasible. I think this is all public knowledge. What you have to do instead is use some sort of proxies as a first pass to nominate a video to even be fed into the machine learning driven algorithm. So that you're only feeding in like a thousand nominees. So the real difference that it can make if you've made a really good video, between it getting to the people who would like it and not getting there. It's not the flaws in the algorithm. The algorithm is probably quite good. It's the mismatch between the proxies being used to nominate stuff to see whether it's even in the running. One of the things used for nomination is understanding the co-watch graph where if you've watched video A and you've also watched video B and then I watch video A. ee whether it's even in the running. One of the things used for nomination is understanding the co-watch graph where if you've watched video A and you've also watched video B and then I watch video A. Your watching both of those gives a little link between them, or maybe you and a ton of other people watching both of them gives a little link between such that once I watch video A, B is potentially nominated in that phase because it's recognized that there's a lot of co-watching. That's something that I'm sure is still quite challenging to do scale but it's more plausible to do at scale than like running some massive neural network. And so I think what might have happened is that by having a bunch of co-watching happening on this same pool of videos, all you need is for some of them to have decent reach and get recommended, right? Because then that’s like igniting a pile of kindling where then if others are good, if they're going to give people good experiences, they get not only nominated but then recommended which then kicks back in the feedback loop there. That turns out to be as close to a guarantee as you can get of saying if you make something that's good, it's a good piece that will satisfy someone, they come away feeling like they learned something that they otherwise didn't know and it was well presented, if you can get it into this peer review process, it will reach people. It's not just going to be shouting into the void And in this case, last year there were over a hundred videos where after the first two weeks they had more than 10,000 views. Which I know is small in the grand scheme but for a fresh channel, talking about a niche mathematical topic, to be able to put it out and get 10,000 people to watch it is amazing. And the idea that that it happen for over a hundred people is amazing That had nothing to do with the prize pool, right? In that the motive might have been a hope of actually getting some reach and having some sense of a guarantee of there being some reach Ironically the reason to do the whole peer review system in the first place is in the service of selecting winners. If you just said “Hey, we're having a watch fest where everyone watches each other's things.” Somehow it wouldn't quite have the same pull that gets people into it. So I think it still makes sense to have winners and to have some material behind those winners. It doesn't have to be much though. And if anything, I think it might ruin it to make it too much. I will also say it's $15,000 actually because we give $500 to 20 different honorable mentions, at least this year. Still pretty modest in the scheme of how much money you can invest to try to get more math lessons in the world. I watched many of the honorable mentions as well because they were just topics that were interesting to me. It's like the thing that the president of Chicago University said. He said we could discard the people we admitted and select the next thousand for our class and there would be no difference. By the way, I really admire not only the education that you have provided directly with your videos which have reached millions of people, but the fact that you're also setting up this way of getting more people to contribute and get to topics that you wouldn't have time to get to yourself. I really admire that you're doing that. If you're self teaching yourself a field that involves mathematics, let's say it's Physics or some other thing like that, there's problems where you have to understand how do I put this in terms of a derivative or an integral and from there, can I solve this integral? What would you recommend to somebody who is teaching themselves quantum mechanics and they figured out how to put how to get the right mathematical equation here. Is it important for their understanding to be able to go from there to getting it to the end result or can they just say well, I can just abstract that out. I understand the broader way to set up the problem in terms of the physics itself.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.