learning-it by example

Back to the video

Embedding Models Compared: 6 Local Models, One Test

The smallest embedding model got 16 of 30. Three others got all 30, and one is a third the size.

Video #48 ยท RAG and vector search

Steps in the video

  1. 0:00 The smallest embedding model got 16 of 30
  2. 0:24 The idea
  3. 0:51 Setup
  4. 1:16 Thirty pages that look alike
  5. 1:43 Six models
  6. 2:15 Score each model
  7. 3:08 What this does not show
  8. 3:32 Mistakes to avoid
  9. 3:59 My take and next steps
  10. 4:31 Recap

Files

  • compare.py (21 lines)
  • models.py (10 lines)
  • shop.py (27 lines)

Download the code (zip)

Tested with

Python 3.12.15 | fastembed==0.9.1

What this does not show

Thirty made-up template pages in English with questions I wrote, one run; three models tied at 30, so this test cannot rank them; speed was not measured.

Sources

Learning IT by Example.