Why AI Gets Dumber With More Context
I used to think more context meant smarter answers. It kind of doesn't, and here is the way I understand why.
I always figured giving a model more context would make it smarter. Feed it everything, get better answers. Turns out that's not really how it works.
These models have huge context windows now, like a million tokens, but that's just how much you can put in, not how much they actually use. The more you stuff in, the more the attention gets spread thin, and stuff starts slipping through. There's this thing people call lost in the middle, where the model pays attention to the start and the end of your prompt and kind of skims everything in between. So the one line that actually mattered, the one you buried in there, just gets ignored.
What got me is that adding more can actually make it worse, even when the extra stuff is relevant. More text is just more places for it to get distracted. The answer is sitting right there and it still misses it.
So now I treat context like a budget, not a bucket. Give it less, but give it the right things. Pull the few bits that matter, keep the prompt short, put the important stuff near the top or the bottom. Honestly the hard part isn't picking what to include. It's picking what to leave out.