learning-it by example

Back to the video

Async AI Agents: Design for Timeouts With Job Polling

In a simulation, a client that retried on a timeout nearly doubled the agent's work, and got half the answers.

Video #239 ยท Enterprise architecture and AI agents

Steps in the video

  1. 0:00 A client that retries on a timeout nearly doubled the work
  2. 0:32 The idea
  3. 1:12 Setup
  4. 1:38 A slow server
  5. 2:00 Two clients
  6. 2:23 Six jobs
  7. 3:22 What this does not show
  8. 3:57 Mistakes to avoid
  9. 4:23 My take and next steps
  10. 4:53 Recap

Files

  • clients.py (24 lines)
  • compare.py (18 lines)
  • jobs.py (18 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

A simulated clock and a pretend server, six jobs I chose; the waiting client gives up after 5 seconds and the poller after 30; the server never cancels abandoned work; no idempotency key; no failures, lost polls or restarts.

Sources

Learning IT by Example.