What is the core idea behind context windows?
The context window is the model's working memory.
How do context windows differ from related concepts?
| Concept | Difference |
|---|---|
| Context Window vs Memory | Context windows are per-session. Memory persists across sessions |
| Context Window vs Knowledge | Knowledge is from training. Context is from the current interaction |
| Context Window vs Token Limit | Token limit includes both input and output within the context window |
How do context windows work?
- User input is converted into tokens
- Tokens fill the context window
- The model processes all tokens in the window simultaneously
- If input exceeds the window, earlier content is truncated or lost
What are the limitations of context windows?
- Information beyond the window is lost
- Long contexts may degrade attention quality
- Larger windows increase compute costs
Why are context windows important?
Context window size determines what tasks an AI model can handle in a single interaction, from short questions to analysing entire documents or codebases.
How are context windows used in practice?
Context window sizes have grown dramatically. GPT-4 supports 128K tokens. Claude supports up to 200K tokens. Google Gemini supports up to 1 million tokens.
Frequently Asked Questions
Does a larger context window mean better performance?
Not always. Research has shown that models can struggle to use information in the middle of very long contexts effectively, a phenomenon known as 'lost in the middle.'
What happens when input exceeds the context window?
Content that exceeds the window is typically truncated, meaning the model cannot see or use that information in its response.