There is a pattern in almost every RAG pipeline that has been through one round of “the answers aren’t good enough.” Someone adds enrichment. A summarizer runs over each chunk, produces a nice abstract, extracts some entities, and writes it all to a description column next to the chunk. The corpus now has metadata. The team feels better.
Then retrieval runs, and the query is embedded, and it is compared against the chunk embeddings — which were computed from the raw chunk text, because that is what was embedded at ingest. The enrichment sits in a column. The retrieval SQL selects doc_id, chunk_index, text. Nothing reads the enrichment. Nothing was ever going to read the enrichment.
This is not a hypothetical failure. It is what a metadata column does by default, and it is invisible, because the enrichment ran, the rows are populated, and a spot check shows lovely summaries sitting exactly where they were written. The pipeline is working. It just is not doing anything.
Put it in the vector, keep the text raw
The interesting move is not to add more metadata. It is to change what gets embedded.
At ingest, a chunk’s vector is computed from the chunk text plus its enrichment — the summary and the named entity terms. The stored text stays the raw chunk. Two different artifacts from one chunk: an embedding that knows more than the text, and a text that is still exactly what the document said.
The consequence is a retrieval behavior that surprises people the first time they see it. A query for “magic items” matches a gear line that never contains the phrase “magic items” — because the enrichment vocabulary participates in the embedding, even though the enrichment never appears in what gets returned. The answer model still receives the raw chunk. It is not reading a summary of a summary. The enrichment did its work at the only moment that matters, which is when the vectors are compared, and then got out of the way.
Storing the raw chunk is not a detail. Enrichment is model output, and model output is wrong sometimes. If you embed the enrichment and store it as the retrievable text, every enrichment error becomes an answer error, permanently, for every future query. Embedding it while storing the original means a bad summary can cost you a retrieval miss — recoverable, measurable — instead of poisoning the answer itself.
The vocabulary problem, which is the actual problem
Enrichment that just restates the chunk in different words buys very little. The reason to enrich is that users do not search in the vocabulary documents are written in.
Documents are written by people who know the subject. Queries are written by people who do not — that is why they are querying. A gear table written by a systems designer says “+2 longsword, attunement required.” The person searching says “magic items.” A benefits handbook says “elective deferral limit.” The employee says “how much can I put in my 401k.” No amount of clever chunking closes that gap, because the gap is not structural. It is lexical.
So the per-chunk summary is written deliberately in the vocabulary users search with, naming the entities plainly. And then a second thing, which does more work than it looks like: hypothetical questions per chunk, embedded as their own retrievable units. A synthetic question sits in the index next to the chunk it answers. A real user question is far closer, in vector space, to a synthetic question than to a declarative passage of source text. Query-to-question beats query-to-document, so you manufacture the questions and let the geometry do the rest.
Some questions have no chunk to land on
Chunk-level retrieval has a category of question it structurally cannot answer, and no amount of top-k tuning fixes it: questions about the document as a whole.
“How many magic items are in here?” has no correct chunk. The information is distributed across forty of them. Retrieval returns the five that look most like the query, the model counts what it can see, and confidently reports five. It is not hallucinating — it answered honestly from what it was given. The retrieval was the failure.
The fix is to manufacture the missing chunk. A reduce pass aggregates the per-chunk enrichment into document-level artifacts — a structured abstract, an entity list, an extractive glossary — and embeds each as its own synthetic retrievable unit. Now “how many magic items are in here” lands on the entity-list chunk, which enumerates every item in one place, and the model can simply count them. The question became answerable because something now exists in the index that answers it.
The glossary definitions are extractive, not generated: each is a sentence lifted verbatim from a cited chunk, validated by substring check against its source. That constraint exists for the reason stated above — enrichment errors are permanent, they poison retrieval and every future answer — so the one artifact most likely to be quoted back to a user is the one artifact that is never allowed to be invented.
Map-reduce, never one huge context
There is a hard constraint underneath all of this, and it is the difference between a technique and a product.
The whole raw document is never placed in a single model call. The map stage enriches each chunk independently — and one chunk fits in any context window, including an 8k local model. The reduce stage aggregates only the small distilled outputs: summaries, term lists, one-line definitions. Never the source.
The lazy version of document-level enrichment is to throw the entire document into a long-context frontier model and ask for an abstract. It works in a demo. Then it meets a 400-page contract set, or a customer whose documents cannot leave their infrastructure, and it is over — you have built something that requires the largest available context window and a cloud API, forever, for every re-index.
Structuring it as map-reduce means the enricher is a pluggable seam: prompt in, text out. Cloud deployments point it at a hosted model. Appliance and air-gapped deployments point it at the resident local model. The orchestration is the same code either way, and it only ever asks the backend for one small completion at a time. Enrichment quality degrades gracefully with model size instead of falling off a cliff when there is no frontier model to call.
Enrichment that fails silently is worse than no enrichment
One last piece, learned the way these things are usually learned. A transient upstream error — a rate limit on a global endpoint, say — can blank enrichment across a whole ingest run. Every row still gets written. Every chunk still gets embedded. The corpus looks complete, and it retrieves worse than it did last week, and nothing anywhere says why.
So enrichment status is recorded per document: off, success, or partial. A corpus whose enrichment silently degraded is detectable, not invisible. This is the least clever paragraph in this post and probably the one that saves the most time, because the failure it prevents is the kind you otherwise find three weeks later while investigating a completely unrelated complaint about answer quality.
What this actually buys
Enrichment done this way is not a quality knob you turn up. It changes what is retrievable:
- Chunks become findable through vocabulary they do not contain.
- Questions match manufactured questions instead of declarative prose.
- Whole-document questions get a chunk that can answer them.
- The returned text stays exactly what the source said.
None of it requires a bigger embedding model, and all of it happens before the first query is ever asked. Which is the general shape of retrieval work worth doing: the expensive, high-leverage decisions are made at ingest, and everything you do at query time is negotiating with choices you already made.
If your pipeline has a description column, it is worth ten minutes to go and read the retrieval SQL and find out whether anything selects it.