Evidence receipt / belief
Published · transcript-backedNathan Lambert: belief
11 Jan 2024 Latent Space RLHF 201 - with Nathan Lambert of AI2 and Interconnects
“I think the things that people see now is like the small models don't really handle nuance as well and they could be more repetitive even if they have really good instruction tuning.”
Source trail
Everything needed to verify it.
- Speaker
- Nathan Lambert
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 11 Jan 2024
- Publisher
- Latent Space
Transcript context
…From text completion to actually chatting back and forth. Yeah. Yeah. Instruction tuning can be multi-turn. Just having a little bit of data that's like a couple of turns can go a really long way. That was like the story of the whole first part of the year is like people would be surprised by how far you can take instruction tuning on a small model. I think the things that people see now is like the small models don't really handle nuance as well and they could be more repetitive even if they have really good instruction tuning. But if you take that kind of 7 to 70 billion parameter jump, like the instruction tuning at the bigger model is like robustness, little things make more sense. So that's still just with instruction tuning and scale more than anything else. Excellent. Shall we go to technical overview?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.