#26 · AI agents and MCP · 5.8 minPlan, Act, Reflect: Does It Help a Small Model?
One example in the prompt took a small model from 2 of 6 to 6 of 6. Reflection added nothing.
Python 3.12.15 | ollama==0.6.3, pydantic==2.14.130 videos tagged "ollama".
#26 · AI agents and MCP · 5.8 minOne example in the prompt took a small model from 2 of 6 to 6 of 6. Reflection added nothing.
Python 3.12.15 | ollama==0.6.3, pydantic==2.14.1
#28 · AI agents and MCP · 5.3 minI gave two MCP servers a tool called search each. Merge their tools by name, and one of them vanishes.
Python 3.12.15 | mcp==2.3.0, ollama==0.6.3
#36 · AI agents and MCP · 5.3 minTwo agents argued, and did no better than a plain vote, for more calls.
Python 3.12.15 | ollama==0.6.3
#39 · AI agents and MCP · 5.2 minI cut my chat to the last three turns and the model forgot all five facts.
Python 3.12.15 | ollama==0.6.3
#43 · RAG and vector search · 5.7 minAn early version of my RAG check passed the answer 'I don't know'. The fixed version still gets things wrong.
Python 3.12.15 | ollama==0.6.3
#45 · RAG and vector search · 4.8 minMy search put the right page first 7 times out of 12. A pretend answer did it 10 times.
Python 3.12.15 | ollama==0.6.3
#46 · RAG and vector search · 5.3 minWith two sentences, text search had the facts for 1 of 6 questions. A graph written by hand had them for all 6.
Python 3.12.15 | ollama==0.6.3
#47 · RAG and vector search · 5.9 minI tried to break a RAG app six ways. Four broke, one barely, and one did not break in five tries.
Python 3.12.15 | ollama==0.6.3
#51 · RAG and vector search · 6.2 minSame filter, same users: applied after the search it found the user's own note 18 times out of 36; applied before the search, all 36.
Python 3.12.15 | ollama==0.6.3
#52 · RAG and vector search · 5.7 minEvery citation was real and every quote exact. With the right source removed, two still pointed at the wrong source.
Python 3.12.15 | ollama==0.6.3
#53 · RAG and vector search · 5.4 minWith a loose threshold, a semantic cache answered 'Do you ship to Norway?' with the answer for Ireland.
Python 3.12.15 | ollama==0.6.3
#54 · RAG and vector search · 5.4 minPasting everything into a 4,096 token prompt got four of four right, until the catalogue doubled.
Python 3.12.15 | ollama==0.6.3
#57 · LLM engineering and evals · 5 minI swapped the order of two answers and my judge changed its mind on half the questions in one test.
Python 3.12.15 | ollama==0.6.3
#58 · LLM engineering and evals · 5.2 minJSON mode with no field names in the prompt fit my fields 0 times out of 12. With the keys named: 12 of 12.
Python 3.12.15 | ollama==0.6.3, pydantic==2.14.1
#60 · LLM engineering and evals · 5 minIn a test with injected failures, retries fixed a simulated busy server, then made twelve attempts and fixed nothing.
Python 3.12.15 | ollama==0.6.3
#61 · LLM engineering and evals · 5.3 minI sent 1,086 tokens into a 1,024-token context on purpose. No error, and the first rule was not followed.
Python 3.12.15 | ollama==0.6.3
#63 · LLM engineering and evals · 5.6 minZero examples scored 12 of 20. Eight scored 18. Flipped labels dropped it to 13.
Python 3.12.15 | ollama==0.6.3
#64 · LLM engineering and evals · 5.1 minOn five multi-step problems my model got 0 right answering directly and 4 right thinking step by step.
Python 3.12.15 | ollama==0.6.3
#68 · LLM engineering and evals · 5 minI asked a small model for 27 test questions and only 17 passed three cheap checks.
Python 3.12.15 | ollama==0.6.3
#69 · LLM engineering and evals · 6.1 minOne check caught 1 of 8 made up answers. A strict model judge flagged 6 of 8 good ones.
Python 3.12.15 | ollama==0.6.3
#87 · Local and open models · 5.5 minOne model, one laptop: 1.6, 6.6 and 9.2 tokens per second. All three were real.
Python 3.12.15 | ollama==0.6.3
#88 · AI security and governance · 5.3 minA sentence I planted in a made-up note told my assistant what to say, and it said it.
Python 3.12.15 | ollama==0.6.3
#89 · AI security and governance · 5.4 minMy bot refused a plain question, then answered four of eight wrapped versions of it.
Python 3.12.15 | ollama==0.6.3
#92 · AI security and governance · 5.2 minMy demo notes agent could delete every note it held, and nothing in the code would stop it.
Python 3.12.15 | ollama==0.6.3
#93 · AI security and governance · 5.5 minA planted page reached the top three results for four of five questions, and a filter missed two leaks.
Python 3.12.15 | ollama==0.6.3
#94 · AI security and governance · 4.9 minI told my bot never to reveal its instructions, put a code in them, asked eight ways, and five worked.
Python 3.12.15 | ollama==0.6.3
#95 · AI security and governance · 5 minAll ten of my model replies passed every check, and three were still wrong.
Python 3.12.15 | ollama==0.6.3
#96 · AI security and governance · 5 minMy banned-word list blocked a made-up test comment about killing a Python process.
Python 3.12.15 | ollama==0.6.3
#102 · AI security and governance · 5 minFour of six test attacks worked on my demo agent; two were plain delete and send requests.
Python 3.12.15 | ollama==0.6.3
#293 · RAG and vector search · 5.2 minI asked a model five questions about a cafe, and it got none right, until I gave it the facts.
Python 3.12.15 | ollama==0.6.3