Evidence receipt / belief
Published · transcript-backedNathan Lambert: belief
11 Jan 2024 Latent Space RLHF 201 - with Nathan Lambert of AI2 and Interconnects
“I think if the people are kind of locked into using synthetic data, people also think that synthetic data is like GPT-4 is more accurate than humans at labeling preferences.”
Source trail
Everything needed to verify it.
- Speaker
- Nathan Lambert
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 11 Jan 2024
- Publisher
- Latent Space
Transcript context
…So do you think like the open source community should spend more time figuring out how to reuse the data that we have or like generate more data? I think that's one of the- I think if the people are kind of locked into using synthetic data, people also think that synthetic data is like GPT-4 is more accurate than humans at labeling preferences. So if you look at these diagrams, like humans are about 60 to 70% agreement. And we're like, that's what the models get to. And if humans are about 70% agreement or accuracy, like GPT-4 is like 80%. So it is a bit better, which is like in one way of saying it. Humans don't even agree with humans 50% of the time.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.