Every model card now advertises a million-token window, but in practice I keep hitting a wall around 150-200k where recall on needle-in-haystack questions drops off badly. Gemini 2.5 Pro holds up better than most for me; Claude degrades more gracefully on code but loses earlier conversation turns; GPT-class models get expensive fast once you cross 128k. Curious whether people are actually shipping 500k+ prompts in production, or if this is mostly a benchmark flex
1M context windows: who actually uses them past 200k?
Replies
No comments yet — be the first to share your thoughts.