learning-it by example

#293 RAG and vector search 5.2 min

RAG for Beginners: The Idea, Then 37 Lines of Python

I asked a model five questions about a cafe, and it got none right, until I gave it the facts.

1. In retrieval augmented generation, what are the four steps?
2. In one run of a small Python RAG test about a made-up Maple Cafe, with llama3.2 on Ollama, how many of the 5 questions did the model get right without retrieval, and how many with it?
3. In a small Python RAG test about a made-up Maple Cafe, why is the model also asked about gluten free brownies, which no fact mentions?
4. In a small Python RAG pipeline, how does the retrieve function choose which facts to paste into the prompt?
5. Which of these is a stated limit of a small Python RAG test with eight facts and five questions?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Look at what is retrieved

    Copy the code folder first. The code needs the Ollama app with all-minilm and llama3.2, as in the video. Write a short script that imports retrieve from rag.py and prints retrieve(question) for each question in facts.py. Predict which fact comes first for each one. On a test run the first fact was the matching fact for all 5 questions.

    A hint

    The video says to look at what was retrieved before blaming the model.

  2. Retrieve only one fact

    In rag.py change the default k=2 in retrieve to k=1. Predict what changes in the prompt for the brownie question. On a test run the one fact retrieved for the brownie question was the flat white price, which does not mention brownies. Then run python compare.py if your machine can run llama3.2 and compare it with the video, or predict the answer if it cannot. Change k back to 2 afterwards, because the next exercise looks at the second fact.

    A hint

    Retrieval always returns the closest facts, even when none of them answers the question.

  3. Add the missing fact

    In facts.py add a ninth fact, The Maple Cafe sells gluten free brownies on Fridays. Predict which fact retrieval will now put first for the brownie question. On a test run it came first, with the flat white price second. Then predict how the last line of compare.py changes.

    A hint

    The brownie question is no longer outside the facts, so the model should no longer need to say it does not know.

  4. Weaken the rule

    In rag.py edit the prompt in ask so it no longer tells the model to say it does not know. Predict what the model will do with the brownie question. This is a prediction because the answer comes from the language model. Compare it with the video's mistake of not telling the model what to do when the facts do not answer.

    A hint

    The rule to remove is the second sentence, the one that begins If they do not answer it. Keep Use only these facts. The model may still decline in other words, and compare.py only looks for do not know or don't know.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Eight made-up facts, five easy questions and one question the facts cannot answer; one seed, one small model (llama3.2), two facts retrieved. An answer counts as right if it contains one expected word. Run on one laptop; your counts may differ.

Up next in RAG and vector search

More in RAG and vector search

All RAG and vector search videos

Related videos