learning-it by example

Back to the video

Read APIs Reliably: Pagination, Retries, Rate Limits

A local API fails 3 times on purpose, and the reliable client still gets all 23 items in 8 requests.

Video #142 ยท Python for data

Steps in the video

  1. 0:00 Three failures on the way, and all 23 items arrive
  2. 0:33 The idea
  3. 1:01 Setup
  4. 1:26 A server that fails on purpose
  5. 1:52 A good page
  6. 2:05 A free port
  7. 2:20 One request, with retries
  8. 2:52 Follow next, with a limit
  9. 3:10 The shortcut
  10. 3:26 The run
  11. 4:54 What this does not show
  12. 5:22 Mistakes to avoid
  13. 5:55 My take and next steps
  14. 6:14 Recap

Files

  • client.py (44 lines)
  • run.py (33 lines)
  • server.py (47 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

A toy local API with planned failures; it does not simulate dropped connections or timeouts, handles only the seconds form of Retry-After, and measures no timing.

Sources

Learning IT by Example.