observation · 13 Feb 2026 · 44:41

qualitative degradation occurs when AI models are trained on short contexts but serve long ones

If you train at a small context length and then try to serve at a long context length, maybe you get these degradations.

Watch at 44:41