Multi-User RAG: Filter Before You Search (Leak Test)
Same filter, same users: applied after the search it found the user's own note 18 times out of 36; applied before the search, all 36.
Video #51 ยท RAG and vector search
Steps in the video
Files
- docs.py (47 lines)
- leak.py (25 lines)
- recall.py (14 lines)
- store.py (30 lines)
Tested with
Python 3.12.15 | ollama==0.6.3
What this does not show
36 made-up notes, 6 users and 4 bait questions, one small model (llama3.2), one run at temperature 0. The search is a plain sort, not a vector database, and the 18 of 36 comes from my design, where every user types the same question. No prompt injection was tested.
Sources
Learning IT by Example.