learning-it by example

#03 RAG and vector search 6.3 min

pgvector vs Chroma vs Qdrant: Same Data, Real Results

Same notes, same question, three vector databases on your laptop: pgvector, Chroma and Qdrant, with real output and three classic mistakes.

1. How do pgvector, Chroma and Qdrant differ in the way you use them?
2. The same question about returning a bike gave pgvector and Chroma a distance of 0.19 and Qdrant a score of 0.81. What does that show?
3. Which distance does Chroma use unless you ask for cosine, and what best distance did the bike returns question get with it?
4. With an HNSW index on 10,000 random vectors, which ORDER BY let Postgres use the pgvector index?
5. Why can the comparison of pgvector, Chroma and Qdrant not tell you which store is fastest?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Ask a different question

    Copy the code folder first. In compare.py change the question to ask when the shop is open on a Saturday. Before you run it, read notes.txt and predict which note each of the three stores will return, and whether the three will agree. Then start the containers as in the video, run python compare.py, and compare your prediction with what you see. This needs Docker and the Ollama embedding model, as in the video.

    A hint

    The note that answers an opening hours question starts with its title, so look for it in notes.txt.

  2. Turn off cosine in Chroma

    In compare.py remove the metadata argument that sets hnsw:space to cosine from the Chroma collection, and put the question back to the bike returns one. Predict the distance Chroma will print compared with pgvector. Then run it and write down the two numbers. Which distance is Chroma using now? Start the Docker containers as in the video first, and stop and remove them when you finish.

    A hint

    The video says that unless you ask for cosine, Chroma measures distance a different way, called L2.

  3. Print Qdrant as a distance

    Qdrant prints a score where higher is closer. Change the print line in with_qdrant so it prints 1 minus hit.score and says lower is closer, like the other two lines. Using the video's idea that similarity is one minus the distance, predict the number it will print before you run it. Start the Docker containers as in the video first, and stop and remove them when you finish.

    A hint

    Compare your prediction with the distance that pgvector and Chroma print for the same question.

  4. Send the right sized vector

    In gotchas.py the Qdrant collection is created with size 384, but the test sends a list of 768 numbers. Change 768 to 384 and predict what the Qdrant part of the output will look like. Think about which lines of the script print anything. Then run it and check. Start the Docker containers as in the video first, and stop and remove them when you finish.

    A hint

    The print about Qdrant refusing a vector sits inside the except branch.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

One question over seven made-up notes on one laptop; no test of speed or scale. The query-plan result is from 10,000 random vectors with an HNSW index. Behaviour is for the pinned versions. Which-to-pick advice is from blog posts, not a benchmark.

Up next in RAG and vector search

More in RAG and vector search

All RAG and vector search videos

Related videos