Technological progress in context length will follow an exponential trajectory, similar to historical trends in computing memory.
We will find out how to use better technology. So I can't really imagine what it's like right now for context links to go out to the billion someday and they might not literally go there, but effectively it'll feel like that. But I know we'll use it and really not wanna go back once we have it.