learning-it by example

#238 Enterprise architecture and AI agents 5.6 min

Event-Driven AI Agents: Queues, Crashes and Duplicates

After a crash, my toy agent wrote two replies to the same ticket. The agent was fine.

1. In an event-driven agent built on a durable log, why can a crash cause the same event to be handled twice?
2. In the crash and restart test with 12 events, what did the plain agent and the idempotent agent end up with?
3. How does the idempotent version of the agent avoid writing a duplicate reply after a restart?
4. What does the agent do with a poison event that always fails, so that one bad event cannot block the rest?
5. In the Python event-driven agent built on a SQLite event log, why does the idempotency check work only for its own example, and what would an email or payment need instead?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Find the worst moment

    Copy the code folder first. Open consumer.py and find the three things that happen for each event: the work, the check for the crash, and the line that saves the position. Write down in which order they run and explain in one sentence why a crash between the work and the saved position makes an event repeat.

    A hint

    The saved position is the log.ack line.

  2. Take away the poison event

    In compare.py, replace the line EVENTS[6] = ("poison", "garbled") with the single word pass, then run python compare.py. Predict the reply count and the dead letter count for each agent first. When I ran it the plain agent showed 13 replies with 1 duplicate and 0 dead letters, and the idempotent agent showed 12 replies with 0 duplicates and 0 dead letters. Then put the EVENTS[6] line back as it was before the next task.

    A hint

    Every one of the 12 events can now be read.

  3. Crash later in the log

    In compare.py, change crash_at=4 to crash_at=9 and predict whether the counts change, then run python compare.py. When I ran it the numbers were the same as before: 12 replies with 1 duplicate for the plain agent and 11 replies with 0 duplicates for the idempotent agent.

    A hint

    Only the one event that was worked on before the crash is repeated, and event 9 is an ordinary ticket. A crash on the poison event 7 would instead repeat its dead letter.

  4. Change the retry limit

    In consumer.py, change attempts=3 to attempts=1 in the definition of run. Predict whether the counts change, then run python compare.py. When I ran it the counts were the same as before, because the poison event always fails and ends in the dead letters whatever the limit. Then write one sentence on what a retry limit protects against.

    A hint

    The poison event fails every time, so more tries never help it.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

One consumer and one faked crash in one process; Redis was not run and consumer groups were not built; the duplicate check works only because the reply is stored next to the log; SQLite is for one machine.

Up next in Enterprise architecture and AI agents

More in Enterprise architecture and AI agents

All Enterprise architecture and AI agents videos

Related videos