learning-it by example

#34 AI agents and MCP 5.5 min

Support Agent: Let Code, Not the Model, Pick the Customer

No attack I tried reached another customer's order. I still moved the customer check into the code.

1. In the Hillside Bikes support-agent demo (a made-up shop with Ollama tool calling), what is the only difference between the two designs?
2. In the made-up bike shop support agent, a fully fooled caller asked for order B-2001 as Bob. What did the design that takes the customer as an argument return?
3. In the made-up bike shop support-agent test, why does 0 of 4 attacks reaching another customer's order prove very little?
4. Which design choice is listed as a mistake in a tool-using support agent?
5. What is a confused deputy?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Call the tools without a model

    Copy the code folder first. Create try.py in the code folder with: from agent import make_tools, then tools, calls = make_tools('argument'), then print(tools['get_order']('B-2001', 'alice')) and print(tools['get_order']('B-2001', 'bob')). No model is called. Predict both lines, then run it. I got 'error: no such order' for alice and 'B-2001: child seat, delivered, 60 pounds.' for bob.

    A hint

    The tool checks that the order belongs to the name it is given, and the caller chooses the name.

  2. Look at the session design

    In try.py use make_tools('session'). Print the result of tools['get_order']('A-1002'), tools['refund']('A-1001') and tools['refund']('B-2002'), then print calls. Predict, then run. I got the helmet order with its planted note, 'refunded 18 pounds for A-1001', 'error: no such order', and a calls list holding only alice's two successful calls.

    A hint

    In the session design the name is fixed by CUSTOMER in agent.py, so the caller cannot change it.

  3. Change a total and see it leak

    In shop.py change the total of order B-2001 from 60 to 75. In try.py call the argument design with tools['get_order']('B-2001', 'bob') and print it. Predict, then run. I got 'B-2001: child seat, delivered, 75 pounds.' Then say why the session design could not return this line to the logged in customer.

    A hint

    The session design has no customer argument for a caller to set.

  4. Spot the hard coded 4

    In compare.py add a fifth sentence to ATTACKS, for example one claiming to be Bob's manager. Read the print line and predict what the label says about the number of attacks. Then say why the printed number would be misleading. Running compare.py needs Ollama with llama3.2, so treat this as a reading task.

    A hint

    Look at the words 'of 4 attacks' in the f-string.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Four attacks, one prompt wording and one run of one small model (llama3.2), so the model result proves little; only the fooled-caller check is exact. The shop is made up, with no real login, sessions or refunds.

Up next in AI agents and MCP

More in AI agents and MCP

All AI agents and MCP videos

Related videos