Alessio Fanelli: evaluation

Transcript context

…So for V1, Edward Beeching and Lewis Tunstall, who were our reinforcement learning team at the time, basically wanted to look at the scores which were there in every paper. So they took all the big RL papers of the time and they looked, and you had GSM 8K, you had MMLU, you had ArcChallenge, systematically. So they took those benchmarks because, yeah, they were kind of obvious which ones were the standards of the time. And when we added evaluation, I think we actually added GSM 8K later on. We also tried to add Drop, which we dropped because there was an implementation problem. When we did our round two, we based ourselves, like not the V2, but I'm going to say V1.5. We interacted a lot with the community to see what was missing in terms of evaluations, what were the capabilities that people wanted to see. And we added those datasets at the time. We keep interacting a lot with the RLHF team, like Lewis Tunstall specifically has been very helpful in helping us choose the last set of evaluations for the V2 also. In the V2 announcement blog posts, you mentioned some of the issues that all their benchmarks had. I feel like it's funny to me that now everybody's bringing up these issues, but before they were just using the numbers for marketing and promotion saying, look how good we do. But now it's like, no, actually the benchmark is wrong, like it should be harder. Are people just finding out recently about these problems because now the scores are getting so high that you're actually inspecting the benchmarks and maybe in the past you were scoring so badly that maybe you weren't as worried about the overall quality? Or what do you think now, like, you know, the last maybe like two, three months have been really like where the leaderboard and like has been kind of like taken off as far as popularity and then why it's now the right time to do V2. And then we'll talk about what V2 actually is. For the first question, when you read evaluation papers, actually a lot of the datasets from, I'm going to say it's a pre-LLM period, are datasets which have been turked basically. So datasets which were made by people which are underpaid, where English is usually not the native language. And so a lot of them have a lot of mistakes and it's kind of obvious just from reading the paper that there are going to be issues because when you generate 10,000 samples, it's quite hard to manually verify each and every one of them. The datasets we've been using, such as MMLU, ArcChallenge and stuff, were of higher quality from the start, but the attention which was given to them forced people to at one point really explore the datasets to see where the scores were coming from. And yes, at the moment, like when a benchmark reaches saturation, so when models basically get the same performance as humans on a benchmark or go above human performance, which the press really likes to say, what it actually says is usually that the models are completely contaminated on said benchmarks and they are now doing errors which humans would not be doing. For example, for MMLU, human performance is at 80-something, also because a number of the questions that humans fail are actually wrong. So the correct answer to those questions is actually a wrong answer. And so the humans who got a wrong answer on this actually had the correct answer. And so if you're getting above humans on this, you have actually learned to predict very wrong stuff, which I find absolutely fascinating. So evaluation has good enough signal-to-noise ratio, if the quality is high enough, it's going to be useful enough for a period of time, and once you reach saturation, you want to inspect it more.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.