Andy Matuschak: evaluation

Transcript context

…You're halfway through a chapter and you're thinking, what is the chapter about? Ok, the broader question is — now that we have all these online resources, some of which you’ve helped develop like Khan Academy, it seems that the value of conscientiousness as a trait has dramatically increased. If you can motivate yourself to learn these things, the world is out there for you to absorb. What are the sort of design or UI or even content modifications that can be made to give you a conscientiousness boost? In the past you had a professor, you had peers, you had in-person deadlines to motivate you. Is there something equivalent to a pen and paper and how that boosts your mathematical IQ for conscientiousness? Right. One enduring result in education psychology is that when you're doing a lot of cognition, metacognition is difficult. What I mean by that is when you're thinking really hard about the stuff on the page, it's very difficult for you to plan, to regulate yourself, to figure out what the best next action to do is, to reflect and evaluate on whether you're understanding things. All that gets harder as the material gets harder and as it gets less familiar. So one common thread, at least in learning science stuff, has been outsourced metacognition. Some of the ways we outsource that are actually very familiar, they're things like somebody gives you a syllabus and tells you what to read when and you reference that. That is a user interface, that is a design practice. If you're a self-motivated student, one thing you can do and that I've done, is just go appropriate a syllabus from some graduate level course that corresponds to the text that you're reading as that might be a good guide to what's most important and how to approach this. There are also lots of things that one can build directly into the interfaces. Just as one example, in Quantum Country, which was a textbook that Michael Nielsen and I developed to explore some ideas around augmented reading experiences, we embedded a bunch of review questions every 1500 words or so in this text on quantum computation. Our primary intention in doing this was to help people remember what they read. We had this theory that part of what makes it hard to learn a complex subject is that there's all these new definitions and notation and terms and things being thrown at you at once and you're being asked to combine these things, which are still unfamiliar. And so you're constantly having to retrieve these elements and struggling to do it, either it's taking a while or your success rate is low. That was our motivation but it had this other metacognitive benefit that was really important. When you’re asked these questions after reading 1500 words it is an opportunity for you to notice that you did not in fact absorb what was in that thing. Not that you don't remember but that there's a word in the question that is apparently important that you simply didn't even notice. And so not only does that give you feedback, it tells you that maybe you need to reread that specific section, but it may also change your behavior towards future sections. In interviews, readers told us that after they reached the first set of questions or a particularly difficult set of questions, they found themselves slowing down and reading more attentively or actually realizing that their reading practices were ineffective in general. In the way that you were mentioning towards the start of the conversation. There's been a bunch of research on adjunct questions, questions that go along with a text, and they have all kinds of effects. the way that you were mentioning towards the start of the conversation. There's been a bunch of research on adjunct questions, questions that go along with a text, and they have all kinds of effects. The adjunct questions have the kind of effects on forward material I was just describing and they also have the effect of making you reflect on what you've just learned. And in addition to the questions being asked, you might find yourself pondering, “Well, I'm being asked about this. But why does this matter?”…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.