Chunk Overlap — AI Glossary

Reference details

Related terms

• Text Splitter
• Parent Document Retrieval
• Naive RAG

How it works

1. After a chunk is cut, the last N tokens are copied to the head of the next, so each chunk advances by size minus overlap rather than by size.
2. A sentence crossing a boundary therefore appears complete in at least one chunk instead of being truncated in both.
3. Chunk count rises as length / (size - overlap), so storage and embedding cost scale with the ratio rather than with the absolute overlap.
4. Retrieval now returns near-duplicate neighbours, since adjacent chunks share text, which is why a deduplication or diversity step often follows.
5. The overlap is blind to meaning: it patches boundaries wherever they fell, which is why it complements a structure-aware splitter rather than replacing one.

Example: The ratio sets the cost, not the overlap

A 50,000-token corpus at 500-token chunks is 100 chunks with no overlap. At 10% overlap each chunk advances 450, giving ceil(50000/450) = 112 chunks — 12% more. At 40% overlap each advances 300, giving ceil(50000/300) = 167, a 67% increase for the same corpus. Doubling the overlap from 20% to 40% does not add 20% of cost; it adds the difference between dividing by 400 and by 300. That non-linearity is why large overlaps get expensive faster than people expect.

Commonly confused with

The parameter that stops a sentence spanning a boundary from being lost to both chunks. It is not a substitute for a splitter that respects structure — overlap patches boundaries blindly, while a structure-aware split avoids creating bad ones. Overlap also multiplies storage and can return two chunks containing the same passage, which crowds the top-k with duplicates.

When to use it

Reach for it when:

• Retrieval returns relevant chunks whose content is cut off, pointing at boundary placement.
• Documents are continuous prose where sentences and ideas cross any fixed cut.
• A structure-aware split is not possible because the source has no usable markup.

Reach for something else when:

• Chunks are already natural units — FAQ pairs, records, short sections — with no boundaries to patch.
• Storage or embedding cost is the binding constraint, where a structure-aware splitter is the cheaper fix.
• Retrieval already returns near-duplicates, which more overlap will worsen.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Chunk Overlap — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path