Evidence receipt / belief
Published · transcript-backedErik Schluntz: belief
28 Nov 2024 Latent Space The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic
“Now, I would say that, you know, this is definitely an area of future research, especially if we talk about these problems that are going to take a human more than four hours.”
Source trail
Everything needed to verify it.
- Speaker
- Erik Schluntz
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 28 Nov 2024
- Publisher
- Latent Space
Transcript context
…this. But still, it's very common to prune bad paths because models get stuck. Yeah. But I'd say that, yeah, 3.5 is not getting stuck as much as previous models. And so, yeah, we wanted to at least just try the most minimal thing. Now, I would say that, you know, this is definitely an area of future research, especially if we talk about these problems that are going to take a human more than four hours. Those might be things where we're going to need to go prune bad paths to let the model be able to accomplish this task within 200k tokens. So certainly I think there's like future research to be done in that area, but it's not necessary to do well on these benchmarks. Another thing I always have questions about on context window things, there's a mini cottage industry of code indexers that have sprung up for large code bases, like the ones in SweetBench. You didn't need them? We didn't.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.