Evidence receipt / belief
Published · transcript-backedAndreas Stuhlmüller: belief
11 Apr 2024 Latent Space Supervise the Process of AI Research — with Jungwon Byun and Andreas Stuhlmüller of Elicit
“I think the answers are actually a little bit clearer on the just kind of basic robustness side of where you can import ideas from normal software engineering and normal kind of DevOps.”
Source trail
Everything needed to verify it.
- Speaker
- Andreas Stuhlmüller
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 11 Apr 2024
- Publisher
- Latent Space
Transcript context
…How do you monitor the ongoing performance of your models? Not to get too LLM-opsy, but you are one of the larger, more well-known operations doing NLP at scale. I guess effectively, you have to monitor these things and nobody has a good answer that I talk to. I don't think we have a good answer yet. I think the answers are actually a little bit clearer on the just kind of basic robustness side of where you can import ideas from normal software engineering and normal kind of DevOps. You're like, well, you need to monitor kind of latencies and response times and uptime and whatnot. I think when we say performance, it's more about hallucination rate, isn't it?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.