learning-it by example

Back to the video

Review AI-Generated Code Fast: Checklist vs Bandit vs Tests

A seven point checklist found seven of eight planted problems in AI style code. A security linter found three.

Video #174 ยท AI coding and productivity

Steps in the video

  1. 0:00 Review it fast
  2. 0:22 The idea
  3. 0:57 Setup
  4. 1:12 The discount rule
  5. 1:30 The rest of the sample
  6. 1:52 The checklist, imports
  7. 2:09 The checklist, patterns
  8. 2:43 The test
  9. 3:34 What this does not show
  10. 4:18 Mistakes to avoid
  11. 4:42 My take and next steps
  12. 5:07 Recap

Files

  • compare.py (49 lines)
  • review.py (49 lines)
  • sample.py (32 lines)
  • test_sample.py (13 lines)

Download the code (zip)

Tested with

Python 3.12.15 | pytest==9.1.1, bandit==1.9.4

What this does not show

No AI model was run: the sample is hand-written with eight planted problems, and the checklist was written knowing them, so the scores show what each reviewer can see, not how often it finds problems in real code. Bandit 1.9.4 ran with default settings; its secret and timeout checks did not match this variable name (API_KEY) or this call (urllib urlopen). Only one logic bug was planted.

Sources

Learning IT by Example.