#46 RAG and vector search 5.3 min
GraphRAG Explained With a Tiny Graph in Python
With two sentences, text search had the facts for 1 of 6 questions. A graph written by hand had them for all 6.
Count the facts by hops
Copy the code folder first. Open a Python prompt in it and import neighbourhood and gold_triples from graph. Predict how many facts neighbourhood("Who manages Sam?", gold_triples(), hops=1) returns, then try hops of 1, 2, 3 and 4 and write down the counts. What happens to the context as you follow the links further?
A hint
Use len() on the result. When I ran it the counts were 2, 4, 8 and 10 out of 13 facts.
A name that is not in the graph
Call neighbourhood("Who is Kim?", gold_triples()). Predict what comes back, then run it. Now add one more triple to a copy of the list, ("Kim", "manages", "Ines"), and ask "Who manages the person who runs the Bristol depot?" before and after. Compare the two lists of facts.
A hint
The code only starts from names that appear in the question. Make the copy with gold_triples() + [the new triple]. In my run, the new fact Kim manages Ines appeared in the list.
How many hops are enough
In compare.py the graph search uses three steps. For each of hops 1, 2 and 3, count how many of the six questions in QUESTIONS get all the documents they need from the hand written graph. Use the holds function from compare.py, copied into your own script so that you do not have to run the model. Predict the three counts first.
A hint
Copy holds into a new script that imports DOCS and QUESTIONS from corpus and neighbourhood and gold_triples from graph. My run gave 1, 5 and 6 questions.
Predict the answers
Without running the model, read the six questions in corpus.py and write down, for each one, the answer you expect and the documents it needs. Then run python compare.py only if Ollama is already installed with llama3.2 and all-minilm. Compare your six answers with the video, where the hand written graph had all the facts for 6 of 6 but the model answered only 3 of 6 right. Which questions might the small model get wrong, and why?
A hint
Questions that need three facts joined are harder for a small model to reason through, even when all the facts are in front of it.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Thirteen made-up sentences, six questions and one small model in one seeded run; a toy graph, not Microsoft GraphRAG (no communities or summaries); text search was held to two sentences while the hand-written graph handed over five to eight of the thirteen.
Up next in RAG and vector search
More in RAG and vector search
- #02 RAG From Scratch in Python: Chunk, Embed, Retrieve (Local)
- #03 pgvector vs Chroma vs Qdrant: Same Data, Real Results
- #40 Chunking Strategies Compared on One Document
- #41 Hybrid Search: BM25 Plus Vectors, Tested
- #42 Reranking With a Cross-Encoder: Does It Help? Python Test
- #43 Evaluate RAG: Recall, Faithfulness and a Wrong Checker
Related videos
- #54 Long Context vs RAG: When Can You Skip Retrieval? RAG and vector search
- #69 Detect LLM Hallucinations: 5 Checks Scored on 16 Answers LLM engineering and evals
- #293 RAG for Beginners: The Idea, Then 37 Lines of Python RAG and vector search
- #52 RAG Citations: Source Removed, Exact Quote, Wrong Source RAG and vector search
