Andrew Gordon Wilson: evaluation

Transcript context

…There's this fundamental trade off between bias and variance. And it feels like you're saying you can have your cake and eat it. And you can keep them in the mixture and you still win. And that just goes against most people's intuition. So I think the bias variance trade off is an incredible misnomer. There doesn't actually have to be a trade off. Folks, that interview with Andrew was absolutely amazing. Keith came over to The UK and we did it in my home studio together a few weeks ago. It's been on Patreon for a little while and I updated so much based on that interview. Andrew was absolutely brilliant. So I know you're gonna love it. But before we kick off, you've probably heard that human data is kind of the dirty secret of Silicon Valley. You know, human data is the reason why these AI models work so well. Because the open AIs and the Anthropics, what they do is they hire humans to do things like data securation and evaluation and post training. And there is a ridiculous kind of uplift from using this human data. Our sponsor, Prolific, what they wanna do is produce the first survey on how human data is being used in AI. And if you volunteer and fill out this form for them, they will give you first access to see how you compare. So, I'd really appreciate it if you did that. There's no personally identifiable information. Link is in the description. And we are also sponsored by Twofer AI Labs. They are an incredible research lab based in Zurich. They've just upgraded their office. They've got an amazing new office. They've hired 13 research engineers in the last year, doing things like reasoning and the ARC challenge. You've probably seen some of the papers that they've published on that. But they have ambitions to build their own foundation models from scratch. They've got an amazing culture. And Benjamin Crusier, the the director, is also very interested in AI safety. So he's going through the Yudkowsky book at the moment. So if that seems like a fit for you, please get in touch with Benjamin Crusier. Go to 2forlabs.ai or look in the description. And also, MLST is sponsored by Cyber Fund. Enjoy the show, folks. Well, Andrew, much of your work challenges conventional wisdom. Is that hard to do? Is there resistance in challenging strongly held beliefs?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.