learning-it by example

#45 RAG and vector search 4.8 min

HyDE and Query Rewriting: 7 of 12 vs 10 of 12

My search put the right page first 7 times out of 12. A pretend answer did it 10 times.

1. What is the HyDE trick for searching with a vague question?
2. In the HyDE and query rewriting demo (a made-up 9-section bike shop handbook, 12 vague questions, llama3.2 with all-minilm search, one run), how many times did each way of searching put the right section first?
3. In the HyDE and query rewriting demo, how is the handbook turned into an index in the search code?
4. In the HyDE and query rewriting demo, why is it fine for the pretend answer to contain made up details?
5. The HyDE and query rewriting demo made its test questions vaguer after a first set of eight easier questions. Why does that matter for the result?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Rank a vague question

    Copy the code folder first. With Ollama running and the all-minilm model pulled, open a Python prompt in the code folder and import rank from search. Predict the top three sections for the question the gears skip when I pedal hard, then print rank("the gears skip when I pedal hard")[:3]. Try two more vague questions of your own and note which section comes first.

    A hint

    rank returns the section headings, best match first. When I ran the gears question, Repairs and servicing came first.

  2. A vague question that misses

    Print rank("I changed my mind about the purchase")[:3]. The answering section is Returns. Predict where it will rank, then run it. Next try rank("How long do I have to return a bike?")[:3] and compare. Explain what the first question shares with the Returns section and what it does not.

    A hint

    When I ran it, Loyalty points came first and Returns second for the vague question, and Returns came first for the clear one.

  3. Write a pretend answer by hand

    For the question I changed my mind about the purchase, write two sentences yourself that might appear in a staff handbook and would answer it. Use rank on your two sentences instead of the question. Predict whether Returns moves to first place, then run it. Say why this works in the same way as the pretend answer in the video.

    A hint

    Use words a handbook would use, such as return, days and refund. Your own sentences may be better or worse than the model's.

  4. Run the model helpers

    If Ollama has the llama3.2 model, import rewrite and pretend_answer from rewrite.py and call both on the question the gears skip when I pedal hard. Read what each returns. Predict first whether the pretend answer will be a true statement from the handbook. Each call can take several seconds on a laptop without a graphics card. Then say why you would never show the pretend answer to a user.

    A hint

    The video says the pretend answer may contain made up details. The code uses temperature 0 and a seed, so the output should be the same each time on the same setup.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

A made-up handbook with nine sections and twelve vague questions that I wrote, made vaguer after a first easier set showed almost no difference. One small model, one embedder, one run, prompts not tuned. A difference of one or two questions is small. Clear questions were not tested.

Up next in RAG and vector search

More in RAG and vector search

All RAG and vector search videos

Related videos