learning-it by example

Back to the video

AI Coding Assistant Workflow: Copy, Test, Read the Diff

I let an AI fix three bugs: one fix passed its tests, two failed, and every fix changed five lines. The copy, test and diff workflow that caught it.

Video #20 ยท AI coding and productivity

Steps in the video

  1. 0:00 Let an AI fix three bugs
  2. 0:21 Why check at all
  3. 0:51 Setup
  4. 1:12 The repo and its bugs
  5. 1:34 The tests come first
  6. 2:10 Work in a copy
  7. 2:34 Measure the diff
  8. 3:18 Reading the results
  9. 4:07 What this does not show
  10. 4:33 Mistakes to avoid
  11. 4:57 My take and next steps
  12. 5:20 Recap

Files

  • assist.py (60 lines)
  • chunks.py (6 lines)
  • discount.py (6 lines)
  • leap.py (6 lines)
  • test_chunks.py (9 lines)
  • test_discount.py (9 lines)
  • test_leap.py (13 lines)

Download the code (zip)

Tested with

Python 3.12.15 | ollama==0.6.3, pytest==9.2.0

What this does not show

One small local model standing in for an assistant, three tiny planted bugs, one run. Not a comparison of any named tool. The run prints pass or fail and the diff size; the description of each edit comes from reading the diffs during preparation and the replies were not saved. The survey was run by Sonar, which sells verification tools, and the review guide is from crystl, which sells a coding terminal.

Sources

Learning IT by Example.