#56 · LLM engineering and evals · 5.3 minPrompt Versioning: Compare Changes Case by Case
I changed a prompt to make it better and the score went down. A score alone could not tell me which cases broke.
Python 3.12.15 | ollama==0.6.33 videos tagged "llm evals".
#56 · LLM engineering and evals · 5.3 minI changed a prompt to make it better and the score went down. A score alone could not tell me which cases broke.
Python 3.12.15 | ollama==0.6.3A made-up change dropped the pass rate from 90 to 60 percent, and the pytest gate exited 1.
Python 3.12.15 | pytest==9.1.1
#317 · Forward deployed engineer interviews · 6 minVersion 2 scored 80% against 70%. On 40 cases, I still could not tell them apart.
Python 3.12.15