learning-it by example

#243 Enterprise architecture and AI agents 5.6 min

Reliability Patterns for AI Agents: Bulkhead, Retry, Cache

In a simulation, one busy agent left the support agent with one call served out of twenty.

1. What does a bulkhead do in a platform of AI agents?
2. In the bulkhead test where a reporting agent floods slow calls, how many of 20 support calls were served with one shared pool of 4, and with 2 slots each?
3. With the downstream completely down for 10 requests, how many calls hit it with retries only, and with retries behind a breaker?
4. In the Python cache simulation of 20 price requests, what did keeping an answer for 100 requests instead of 1 request show?
5. What does a real retry need that the simple retry in the Python bulkhead, retry, breaker and cache demo does not have?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Read the price of each pattern

    Copy the code folder first. Run python compare.py. For each of the four patterns, write down one number that shows what it gained and one number that shows what it cost. For example, retry gained answers but made more calls.

    A hint

    Look at the answered and calls made values for retry, and at calls and stale answers for the cache.

  2. Fewer retry attempts

    In compare.py, change the retry test from ("up to 3 attempts", 3) to ("up to 2 attempts", 2). Predict the answered count and the number of calls, then run python compare.py. When I ran it I got 5 of 10 answered and 15 calls made. Say why 2 attempts did not help.

    A hint

    Five of the requests fail on their first two attempts, so they need a third.

  3. Change the breaker dials

    In compare.py, change Breaker() to Breaker(limit=2, pause=10). Predict whether the number of calls that hit the dead downstream goes up or down, then run python compare.py. When I ran it the breaker line showed 6 calls, down from 14.

    A hint

    A lower limit opens the breaker sooner, and a longer pause keeps it open longer.

  4. Make the shared pool bigger

    In compare.py, change Pool(4 if shared else 2) to Pool(8 if shared else 2). The printed label still says a shared pool of 4 because it is fixed text. Predict how many support calls are served with the shared pool, then run python compare.py. When I ran it I got 3 of 20 for the shared pool and 20 of 20 for the bulkhead. Say what this shows about adding slots versus separating them.

    A hint

    The reporting agent uses two slots a second for 5 seconds each.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Simulated faults on a counted clock with settings I chose; in the shared pool the reporting agent always asks first; the retry has no delay or backoff; the breaker has no half-open state and its pause is counted in calls; the cache counts requests, not time.

Up next in Enterprise architecture and AI agents

More in Enterprise architecture and AI agents

All Enterprise architecture and AI agents videos

Related videos