HN Explorer

Ask HN: Why don't we talk about context window size anymore?

2 points by rce123 ↗ HN
Context length used to be one of the headline metrics for new models. I hardly hear about it anymore, but I can't imagine it doesn't still play a huge role in practical performance.

Is this truly no longer a bottleneck, or does it just get rolled up in the headline benchmark numbers now?

4 comments

[ 4.0 ms ] story [ 19.6 ms ] thread
verdverm
The models have reached, on average, the 256k/1M window size and that is more than enough for them to complete most tasks without losing attention. I suspect more people are using fresh sessions and markdown between them, than trying to keep fewer long running sessions - this works better overall. We are moving more into the quality >>> quantity realm.
HeDoesntKnow
> that is more than enough for them to complete most tasks without losing attention

Most inaccurate thing I have heard all day.

If you paste a 1k line React component and ask a frontier model to add a feature or do some refactor, it will do a good job.

Paste 10k lines of code files - no chance it does it right the first try. It will start making all kinds of mistakes.

I use this stuff daily - I don’t care what they advertise here are the literal facts using the best models available today:

1 function = perfect

1k lines of code = good and works but probably not in the exact style you said

5k lines = stuff is getting omitted, assumptions are being made

10k lines = no chance it works

I use LLMs for coding every day