Context & Memory
Engineering the Context Window
Retrieval, memory, and the working set a model actually reasons over
The context window is the one surface you fully control, so most problems blamed on model quality are context-assembly problems.
Retrieval, memory, compaction and ordering are usually built by different people at different times, and then the result is judged as though the model were the variable. This book treats them as a single system for deciding what the model sees: an evidence pipeline feeding a budgeted, ordered window, with provenance, freshness and access control surviving all the way into the prompt. It also takes the long-context argument seriously, since long windows did kill lazy retrieval, and gives you the tests that show which parts of your own pipeline are now redundant.
What it makes operable
- 01Decide retrieval versus long context on corpus size, freshness, access control and cost
- 02Run lexical and vector retrieval together and fuse them by rank rather than score
- 03Assemble a window under an explicit token budget, with decisive evidence out of the middle
- 04Write a memory policy that holds up when the evidence is untrusted or poisoned
Contents
12 of 12 published- 01Context Is the Only Thing You ControlThe surface you own, as against the weights you do not.11 min
- 02What Long Context Fixed, and What It Did NotMid-window accuracy loss and cost per query, measured on your own corpus.10 min
- 03Retrieval as an Evidence PipelineThe stages, contracts and failure points between a query and a supported claim.10 min
- 04Lexical and Vector Retrieval, TogetherWhy exact identifiers still need BM25, and how to fuse two ranked lists.10 min
- 05Rerankers and the Cost of Being SureSpending cross-encoder time only where it changes the evidence set.10 min
- 06Chunking Is a Document-Modelling ProblemSegmenting on structure so a claim keeps the qualifier that bounds it.11 min
- 07Assembling the Window: Order, Budget, CompactionA token budget per section, an ordering rule, and compaction you can reverse.11 min
- 08Memory: What to Write, What to ForgetA write policy for durable facts, with provenance and a TTL on every record.10 min
- 09Provenance, Freshness, and Access Control Inside ContextKeeping who-may-see-this and how-old-is-this attached to the evidence itself.10 min
- 10Poisoned Memory and Untrusted EvidenceTreating retrieved text, including your own memory, as attacker-controlled input.11 min
- 11Measuring Context Quality Independently of the ModelScoring the window so a bad answer can be blamed on the right stage.10 min
- 12Migrating a RAG System Into a Context SystemThe order of operations for rebuilding a plateaued pipeline in place.10 min
Evidence
4 sourcesThird-party sources behind the book's premise. Every figure in them belongs to the party that published it and is attributed to them in the text.
- RAG vs Long Context: What the 2026 Data Shows
Wire · 2026-04-08 · Industry report · reported
- Context Engineering: A Practical Guide for AI Agents (2026)
Sourcegraph · 2026-05-28 · Vendor engineering · reported
- Lost in the Middle: How Language Models Use Long Contexts
Stanford University · 2023-07-06 · Research paper · verified
- Reciprocal rank fusion
Elastic · Official documentation · verified
Continue through the library
