learning-it by example

#39 AI agents and MCP 5.2 min

Context Engineering: What to Keep From Chat History

I cut my chat to the last three turns and the model forgot all five facts.

1. What is context engineering?
2. In the single run with llama3.2 on the made-up ten-turn cycling trip chat, how did keeping the whole chat compare with keeping only the latest 3 turns?
3. In the single run with llama3.2 on the made-up cycling trip chat, which facts did the model summary strategy lose?
4. Why does the video's ten-turn chat test say nothing about very long chats?
5. In the chat-history demo, how does the pinned facts strategy build the prompt?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. See what each strategy sends

    Copy the code folder first. No model is called for this. Run Python in the code folder with: from strategies import history, then notes, messages = history('pinned facts', 'x'), then print(notes) and print(len(messages)). I got the two Remember lines as the notes and 6 messages. Then do the same for 'everything' and 'latest turns only' and predict the message counts first. I got 20 and 6.

    A hint

    Each turn is one user message and one assistant message, and KEEP is 3.

  2. Pin one more fact

    In chat.py change the hills turn so the user says 'Remember: I really want to avoid steep hills, my knee is bad.' Run the history check from the first exercise again for 'pinned facts'. Predict what the notes now contain, then compare. Running compare.py to see the model's scores needs Ollama, so treat the right answer counts as a prediction.

    A hint

    Pinned facts are the user lines that start with Remember:.

  3. Keep five turns, not three

    In strategies.py change KEEP from 3 to 5. Read chat.py and write down which of the five facts are inside the last five turns. Predict how many of the five questions 'latest turns only' can answer now, and say why the video's result of 0 of 5 would change.

    A hint

    Count back five turns from the end of the TURNS list.

  4. Write your own test question

    In chat.py add a sixth fact to one of the turns, for example the colour of your bike, and add a sixth tuple to QUESTIONS with a text the answer must contain. Predict which of the four strategies are most likely to keep it, using what you learned about summaries and Remember lines. Note that the print line in compare.py says 'of 5' and also divides the token and second totals by 5, so with six questions its averages will be too high unless you change those 5s to 6.

    A hint

    The video says to test recall with questions whose answers you know.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

A made-up ten turn chat, one run, one small local model, five questions I wrote, scored by whether the reply contains a word. The summary is capped at 80 tokens and I did not check whether the cap or the model dropped the two facts. Ten turns is tiny, so this says nothing about long chats. Timings are from one laptop.

Up next in AI agents and MCP

More in AI agents and MCP

All AI agents and MCP videos

Related videos