learning-it by example

#301 RAG and vector search 7 min

Types of RAG Explained With Examples, Tested

Six kinds of RAG, one small test: the plainest one scored 10 of 12, and the graph one answered a question none of the others could.

1. On the bike shop test of 41 pages and 12 questions, how many questions did plain embedding search get fully right?
2. In a retrieval-only test of six RAG types on a made-up bike shop, how does a type earn credit for a question?
3. In a retrieval-only test of six RAG types on a made-up bike shop, which type got both needed pages for the question about who manages the workshop that services Leeds orders?
4. In a small Python test of RAG types, the linked way (the graph idea) only works when what condition holds?
5. In a test of six RAG types on 12 questions, why is a difference of one question treated as noise?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Guess who answers each question

    Copy the code folder first. The full run needs the fastembed and numpy packages from the video, so these are prediction tasks. Read the twelve QUESTIONS in docs.py with the pages each one needs. Pick two questions you think plain embedding search could miss, and say why. Then compare with the video, where plain got 10 of 12.

    A hint

    Look at the two step question about Leeds and the questions that need two pages.

  2. Find the pages a question needs

    In docs.py, the number in each set is the index of a page in PAGES, counting from 0. For the question Which suppliers provide parts for the X7-200?, the set is {13, 14}. Count down PAGES to find those two pages, and write down what each one says.

    A hint

    Pages 13 and 14 are the Brakeco and GearWorks pages.

  3. Remove a hand-written hint

    In rag.py the HINTS table holds hand-written words that stand in for a language model. Read the rewritten function and predict what happens to the question How much is it to get my ride checked over? when its hint is removed from the table. Compare with the video, where rewrite and plain got the same questions. This is a prediction because the packages are needed to run it.

    A hint

    The hint is added to the question before the search.

  4. Why linked scored zero

    Read the linked function and the named helper in rag.py. A page counts as a link when it names two of the things in THINGS. Pick one plain fact question from docs.py, such as Is cash accepted?, and predict why the linked way cannot give the right page. Compare with the video, where linked scored 0/2 on the fact, code, twins and vague kinds.

    A hint

    The linked way only works when the question names something that is on the list.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Forty-one made-up pages and twelve questions written by the author; one question is a twelfth of the score. Retrieval only: no language model answers anything. Rewrite uses hand-written hints and linked a hand-made list of names. Scores move with settings: two pages handed over, six rerank candidates, fusion constant 60.

Up next in RAG and vector search

More in RAG and vector search

All RAG and vector search videos

Related videos