Chunking Strategies Compared on One Document
Chunking decides what your RAG app can find. I cut one document five ways and scored each against ten questions.
Video #40 ยท RAG and vector search
Steps in the video
Files
- chunkers.py (17 lines)
- compare.py (44 lines)
- cut.py (13 lines)
- golden.py (13 lines)
- handbook.md (28 lines)
Tested with
Python 3.12.15 | ollama==0.6.3
What this does not show
One made-up 586 word handbook and ten questions I wrote, so one question is a ten percent swing: a method, not a ranking. Clean headings favour the heading split. One small embedding model. No semantic chunking tried.
Sources
Learning IT by Example.