Engineering the Context Window
Retrieval, memory, and the working set a model actually reasons over
The context window is the one surface you fully control, so most problems blamed on model quality are context-assembly problems.
Retrieval, memory, compaction and ordering are usually built by different people at different times, and then the result is judged as though the model were the variable. This book treats them as a single system for deciding what the model sees: an evidence pipeline feeding a budgeted, ordered window, with provenance, freshness and access control surviving all the way into the prompt. It also takes the long-context argument seriously, since long windows did kill lazy retrieval, and gives you the tests that show which parts of your own pipeline are now redundant.
What it makes operable
- 01
Decide retrieval versus long context on corpus size, freshness, access control and cost
- 02
Run lexical and vector retrieval together and fuse them by rank rather than score
- 03
Assemble a window under an explicit token budget, with decisive evidence out of the middle
- 04
Write a memory policy that holds up when the evidence is untrusted or poisoned
Contents
12 of 12 published
Every chapter is free to read in the browser, cites its own sources, and stands on its own if you came for one decision rather than the whole argument.
- 01Context Is the Only Thing You ControlThe surface you own, as against the weights you do not.11 min
- 02What Long Context Fixed, and What It Did NotMid-window accuracy loss and cost per query, measured on your own corpus.10 min
- 03Retrieval as an Evidence PipelineThe stages, contracts and failure points between a query and a supported claim.10 min
- 04Lexical and Vector Retrieval, TogetherWhy exact identifiers still need BM25, and how to fuse two ranked lists.10 min
- 05Rerankers and the Cost of Being SureSpending cross-encoder time only where it changes the evidence set.10 min
- 06Chunking Is a Document-Modelling ProblemSegmenting on structure so a claim keeps the qualifier that bounds it.11 min
- 07Assembling the Window: Order, Budget, CompactionA token budget per section, an ordering rule, and compaction you can reverse.11 min
- 08Memory: What to Write, What to ForgetA write policy for durable facts, with provenance and a TTL on every record.10 min
- 09Provenance, Freshness, and Access Control Inside ContextKeeping who-may-see-this and how-old-is-this attached to the evidence itself.10 min
- 10Poisoned Memory and Untrusted EvidenceTreating retrieved text, including your own memory, as attacker-controlled input.11 min
- 11Measuring Context Quality Independently of the ModelScoring the window so a bad answer can be blamed on the right stage.10 min
- 12Migrating a RAG System Into a Context SystemThe order of operations for rebuilding a plateaued pipeline in place.10 min
Evidence · 4 sources
Third-party sources behind the book's premise. Every figure in them belongs to the party that published it and is attributed to them in the text.
In the age of AI
The advantage was never the model. It's knowing what to build with it — and having a team that can actually ship it.
That's the part I help with: finding where AI genuinely makes your business faster, deciding what's worth building, and standing behind it once it's live.
Four offices, one very full passport
Every dot on this map is a conversation I still remember.
- Where I've spoken
- Office
