Chunking turns documents into the units a retrieval system can locate. The choice determines which text arrives together at answer time, which makes it a decision about correctness rather than a preparation step before the real work.
The boundary is the mechanism
Retrieval returns whole units. Whatever the boundary separates, the model does not see together.
Take a fixed-size splitter that places an eligibility rule at the end of one chunk and its exception at the start of the next. A query about the rule retrieves the rule. The answer is fluent, cites a genuine source, and is wrong in exactly the cases the exception exists to cover — which are the cases anyone bothers to ask about.
No chunking approach is established as correct. Fixed-size, content-based, semantic, and combined approaches are all in use, overlap is use-dependent, and a labelled semantic method is not evidence of performance. Character counts are not model tokens, and a nominal character limit can quietly exceed the actual budget.
Testing it
The comparison holds the source snapshot and the evaluation questions fixed, varies one documented chunking choice, and examines four outcomes separately: retrieval coverage, whether the cited location is correct, whether the answer is actually supported by what was retrieved, and operating cost.
Keeping those separate matters because they fail independently. A change can improve relevance while breaking citation fidelity, and a single quality score hides that.
Hold the embedding model, ranking, and generation configuration fixed where possible. Where they must change together, identify the changed dependencies rather than attributing the combined effect to chunking. Separate the questions used during development from a held-out set, and stop retuning against the same questions and presenting the result as performance on unseen work.
Start from a simple baseline and justify each alternative against it. Inspect the misses and, more usefully, the misleading hits — the retrievals that looked right and supported a wrong answer.
The failures worth designing against
Each choice has a characteristic way of going wrong, and these are the ones to construct test questions for.
A boundary rule and overlap setting separates a condition from its exception, or duplicates the same evidence across units so it appears corroborated. Header, table, and caption handling detaches a row from the meaning its header carried, or strips a qualification. Source identifiers and locators produce a citation that resolves to a different revision of the document. Parent metadata and access fields leave a chunk separated from the permission that governed its source, which is a retrieval result someone should not have seen. A rebuilt index becomes incompatible with the consumer that reads it. And the refresh mechanism leaves withdrawn evidence searchable.
The pipeline before the chunker
Ingestion decides what text exists to be chunked, and its failures are invisible downstream.
Extraction is configuration-dependent in ways that change between tool versions: processing images embedded in a document and rendering pages for OCR are different operations that can run together, and behaviour differs across major versions of the same extractor. Difficult pages, tables, footnotes, and image-only content need inspecting against the source, and rejected or partial extractions need retaining rather than dropping.
Generated captions and summaries stay labelled as generated, separate from extracted text. Promoting a generated caption into a primary-source quotation is how a system ends up citing itself.
Deletion is the part that is assumed rather than built. Removing a document from source storage does not remove its chunks, full parent deletion propagation can depend on a configured deletion-detection policy, child keys change during source updates, and manual index edits get overwritten on refresh. A withdrawn policy document whose old chunks remain searchable is a live system serving retracted guidance with a valid-looking citation.
The rule
What stays fixed is that every retrieval unit must carry its source identity, its revision, and its access context. What changes is the segmentation, and changing it invalidates the identifiers that citations and downstream consumers depend on — so migration cost is part of the decision, not a consequence of it.
What transfers
The record the receiving team needs is the segmentation rules, the tokenizer and model versions, the metadata schema, the source-to-chunk mappings, the index configuration, and a rebuild, update, and deletion procedure.
The acceptance test is behavioural rather than documentary. Ask the recipient to rebuild a permitted sample, resolve its citations to the correct revisions, and execute a correction and a withdrawal, without undocumented help. A pipeline that only its author can operate has not transferred.
Not to be confused with
A vendor default. Documented approaches are starting points scoped to their products, and published effort and cost comparisons are labelled subjective by the vendors themselves.
Reproducibility. Exporting an index is not the ability to rebuild one. The rebuild depends on source access, licences, tokenizer versions, and model availability, and any of those can be missing while the export succeeds.