Sarah Saab: belief

Transcript context

…How do you get them interested? The first misconception is that the work is sort of grinding or boring, which actually in our experience when it comes to providing either training or evaluation or fine tuning data for SOTA models, it's not. It's actually very, very deeply interesting and cerebral. And I think the other misconception probably stems from the history of the click working and crowd working space, which is that these people are poorly paid, which is also not true. They're actually paid quite a lot. Duration of task at any 1 time is not very long, so they're not sitting in front, at least on our model, they're not sitting in front of a computer for 8 hours in a row, and we find that you don't get high quality human data by putting someone in front of a computer and asking them to do the same thing for 8 hours straight anyway. So I think the incentives on both sides are aligned in that sense. So it's usually shorter durational work that is often interspersed among other employment or other things that these experts do, but they're paid actually quite well for the time they spend and that's part of our ethical stance. And also, you know, it's a competitive space when when it comes to sort of the experts at the edge of human knowledge. In some cases, there aren't there really aren't that many of them that can contribute something helpful to a frontier model's corpus of knowledge. Like when I spoke with Francois Schollet about the art challenge, he said that when they were getting the human testers, had to be super careful because people have a limited attention span. It's the same with code review for example, like you can't really get people doing code review for more than like half an hour or something because they start rubber stamping and they start, you know, and you so you folks must have done so much research on this.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.