#39 AI agents and MCP 5.2 min
Context Engineering: What to Keep From Chat History
I cut my chat to the last three turns and the model forgot all five facts.
See what each strategy sends
Copy the code folder first. No model is called for this. Run Python in the code folder with: from strategies import history, then notes, messages = history('pinned facts', 'x'), then print(notes) and print(len(messages)). I got the two Remember lines as the notes and 6 messages. Then do the same for 'everything' and 'latest turns only' and predict the message counts first. I got 20 and 6.
A hint
Each turn is one user message and one assistant message, and KEEP is 3.
Pin one more fact
In chat.py change the hills turn so the user says 'Remember: I really want to avoid steep hills, my knee is bad.' Run the history check from the first exercise again for 'pinned facts'. Predict what the notes now contain, then compare. Running compare.py to see the model's scores needs Ollama, so treat the right answer counts as a prediction.
A hint
Pinned facts are the user lines that start with Remember:.
Keep five turns, not three
In strategies.py change KEEP from 3 to 5. Read chat.py and write down which of the five facts are inside the last five turns. Predict how many of the five questions 'latest turns only' can answer now, and say why the video's result of 0 of 5 would change.
A hint
Count back five turns from the end of the TURNS list.
Write your own test question
In chat.py add a sixth fact to one of the turns, for example the colour of your bike, and add a sixth tuple to QUESTIONS with a text the answer must contain. Predict which of the four strategies are most likely to keep it, using what you learned about summaries and Remember lines. Note that the print line in compare.py says 'of 5' and also divides the token and second totals by 5, so with six questions its averages will be too high unless you change those 5s to 6.
A hint
The video says to test recall with questions whose answers you know.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
A made-up ten turn chat, one run, one small local model, five questions I wrote, scored by whether the reply contains a word. The summary is capped at 80 tokens and I did not check whether the cap or the model dropped the two facts. Ten turns is tiny, so this says nothing about long chats. Timings are from one laptop.
Up next in AI agents and MCP
More in AI agents and MCP
- #01 Local AI Agent in Python: No API Key (Ollama)
- #12 Multi-Agent Patterns in Plain Python: Pipeline, Router, Loop
- #22 OpenAI Agents SDK vs LangGraph vs Plain Python
- #23 Agent Memory: What to Keep and What to Forget
- #24 Approval Gates for Risky Agent Actions
- #25 Tool Errors: Retries, Fallbacks and Made-Up Answers
Related videos
- #54 Long Context vs RAG: When Can You Skip Retrieval? RAG and vector search
- #61 Count Tokens Before You Send: Context Limits LLM engineering and evals
- #88 Indirect Prompt Injection: A Note That Gave Orders AI security and governance
- #36 Multi-Agent Debate: Do Two Agents Beat One? A Test AI agents and MCP
