#293 RAG and vector search 5.2 min
RAG for Beginners: The Idea, Then 37 Lines of Python
I asked a model five questions about a cafe, and it got none right, until I gave it the facts.
Look at what is retrieved
Copy the code folder first. The code needs the Ollama app with all-minilm and llama3.2, as in the video. Write a short script that imports retrieve from rag.py and prints retrieve(question) for each question in facts.py. Predict which fact comes first for each one. On a test run the first fact was the matching fact for all 5 questions.
A hint
The video says to look at what was retrieved before blaming the model.
Retrieve only one fact
In rag.py change the default k=2 in retrieve to k=1. Predict what changes in the prompt for the brownie question. On a test run the one fact retrieved for the brownie question was the flat white price, which does not mention brownies. Then run python compare.py if your machine can run llama3.2 and compare it with the video, or predict the answer if it cannot. Change k back to 2 afterwards, because the next exercise looks at the second fact.
A hint
Retrieval always returns the closest facts, even when none of them answers the question.
Add the missing fact
In facts.py add a ninth fact, The Maple Cafe sells gluten free brownies on Fridays. Predict which fact retrieval will now put first for the brownie question. On a test run it came first, with the flat white price second. Then predict how the last line of compare.py changes.
A hint
The brownie question is no longer outside the facts, so the model should no longer need to say it does not know.
Weaken the rule
In rag.py edit the prompt in ask so it no longer tells the model to say it does not know. Predict what the model will do with the brownie question. This is a prediction because the answer comes from the language model. Compare it with the video's mistake of not telling the model what to do when the facts do not answer.
A hint
The rule to remove is the second sentence, the one that begins If they do not answer it. Keep Use only these facts. The model may still decline in other words, and compare.py only looks for do not know or don't know.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Eight made-up facts, five easy questions and one question the facts cannot answer; one seed, one small model (llama3.2), two facts retrieved. An answer counts as right if it contains one expected word. Run on one laptop; your counts may differ.
Up next in RAG and vector search
More in RAG and vector search
- #02 RAG From Scratch in Python: Chunk, Embed, Retrieve (Local)
- #03 pgvector vs Chroma vs Qdrant: Same Data, Real Results
- #40 Chunking Strategies Compared on One Document
- #41 Hybrid Search: BM25 Plus Vectors, Tested
- #42 Reranking With a Cross-Encoder: Does It Help? Python Test
- #43 Evaluate RAG: Recall, Faithfulness and a Wrong Checker
Related videos
- #45 HyDE and Query Rewriting: 7 of 12 vs 10 of 12 RAG and vector search
- #46 GraphRAG Explained With a Tiny Graph in Python RAG and vector search
- #47 6 RAG Failure Modes Reproduced: Which Ones Actually Break RAG and vector search
- #51 Multi-User RAG: Filter Before You Search (Leak Test) RAG and vector search
