#45 RAG and vector search 4.8 min
HyDE and Query Rewriting: 7 of 12 vs 10 of 12
My search put the right page first 7 times out of 12. A pretend answer did it 10 times.
Rank a vague question
Copy the code folder first. With Ollama running and the all-minilm model pulled, open a Python prompt in the code folder and import rank from search. Predict the top three sections for the question the gears skip when I pedal hard, then print rank("the gears skip when I pedal hard")[:3]. Try two more vague questions of your own and note which section comes first.
A hint
rank returns the section headings, best match first. When I ran the gears question, Repairs and servicing came first.
A vague question that misses
Print rank("I changed my mind about the purchase")[:3]. The answering section is Returns. Predict where it will rank, then run it. Next try rank("How long do I have to return a bike?")[:3] and compare. Explain what the first question shares with the Returns section and what it does not.
A hint
When I ran it, Loyalty points came first and Returns second for the vague question, and Returns came first for the clear one.
Write a pretend answer by hand
For the question I changed my mind about the purchase, write two sentences yourself that might appear in a staff handbook and would answer it. Use rank on your two sentences instead of the question. Predict whether Returns moves to first place, then run it. Say why this works in the same way as the pretend answer in the video.
A hint
Use words a handbook would use, such as return, days and refund. Your own sentences may be better or worse than the model's.
Run the model helpers
If Ollama has the llama3.2 model, import rewrite and pretend_answer from rewrite.py and call both on the question the gears skip when I pedal hard. Read what each returns. Predict first whether the pretend answer will be a true statement from the handbook. Each call can take several seconds on a laptop without a graphics card. Then say why you would never show the pretend answer to a user.
A hint
The video says the pretend answer may contain made up details. The code uses temperature 0 and a seed, so the output should be the same each time on the same setup.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
A made-up handbook with nine sections and twelve vague questions that I wrote, made vaguer after a first easier set showed almost no difference. One small model, one embedder, one run, prompts not tuned. A difference of one or two questions is small. Clear questions were not tested.
Up next in RAG and vector search
More in RAG and vector search
- #02 RAG From Scratch in Python: Chunk, Embed, Retrieve (Local)
- #03 pgvector vs Chroma vs Qdrant: Same Data, Real Results
- #40 Chunking Strategies Compared on One Document
- #41 Hybrid Search: BM25 Plus Vectors, Tested
- #42 Reranking With a Cross-Encoder: Does It Help? Python Test
- #43 Evaluate RAG: Recall, Faithfulness and a Wrong Checker
