How to chunk documents for RAG
- Drop text, Markdown, CSV, HTML or JSON files, or paste a document.
- Choose the chunk size and overlap, and whether to measure in estimated tokens, characters or words. 300 to 800 tokens with 10–15% overlap is a common starting point.
- Pick a strategy: recursive splitting keeps paragraphs and sentences together, the Markdown option starts new chunks at headings, and fixed size cuts evenly.
- Review the chunks, then export JSONL, JSON or CSV with an id and metadata for each chunk, ready to embed and load into a vector database.
Choosing chunk size and overlap
Retrieval-augmented generation (RAG) answers questions by finding the most relevant pieces of your documents and passing them to a language model. Those pieces are chunks, and how you cut them has a big effect on answer quality. Chunks that are too small lose context: a sentence like "It must be unused" means nothing without the heading above it. Chunks that are too large dilute the embedding, so a search for a specific detail matches less precisely, and they waste context window space.
Splitting on natural boundaries helps: this tool works like the popular recursive character splitter, trying paragraph breaks first, then line breaks, sentences and finally words, so chunks rarely end mid-sentence. Overlap repeats the end of one chunk at the start of the next so that facts spanning a boundary are not lost. For structured documents, the Markdown option keeps each section together. Token counts are estimated at about four characters per token; use our Token Counter for exact OpenAI counts. Everything runs locally.
What chunk size is best for RAG?
There is no single answer. Start around 400–600 tokens with 50–100 tokens of overlap, then test retrieval on real questions and adjust.
Why use overlap?
It keeps information that spans a chunk boundary retrievable from at least one chunk.
Which export format should I use?
JSONL is easiest for embedding pipelines and vector databases; CSV suits spreadsheets and quick reviews.