#53 · RAG and vector search · 5.4 minSemantic Cache for LLM Apps: Pick the Threshold (Python)
With a loose threshold, a semantic cache answered 'Do you ship to Norway?' with the answer for Ireland.
Python 3.12.15 | ollama==0.6.34 videos tagged "llm cost".
#53 · RAG and vector search · 5.4 minWith a loose threshold, a semantic cache answered 'Do you ship to Norway?' with the answer for Ireland.
Python 3.12.15 | ollama==0.6.3Routing cost 19.1 made-up cents instead of 40, and got one more of 20 questions wrong.
Python 3.12.15
#244 · Enterprise architecture and AI agents · 5.5 minOne agent in a loop used 88 percent of a made-up day's spend, and nothing stopped it.
Python 3.12.15
#318 · Forward deployed engineer interviews · 5.9 minCutting the prompt barely helped. A small model with shorter answers made it fit.
Python 3.12.15