learning-it by example

#20 AI coding and productivity 5.6 min

AI Coding Assistant Workflow: Copy, Test, Read the Diff

I let an AI fix three bugs: one fix passed its tests, two failed, and every fix changed five lines. The copy, test and diff workflow that caught it.

1. In the safe workflow for letting a local AI model fix bugs in a small Python repo, where is the model's corrected file written?
2. How many of the three fixes passed their tests in the AI coding assistant workflow demo?
3. The chunk fix passed its tests. Why did the diff size still matter for it?
4. What does the video say this example does not show?
5. Which of these does the video list as a mistake to avoid?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Fix the chunk bug by hand

    Copy the code folder first. Run python -m pytest -q -p no:cacheprovider and note that 3 tests fail and 4 pass. Then open chunks.py and change the end of the range so that the last partial group is kept (the range should stop at len(items), not earlier). Run the tests again. When we ran this change, 2 failed and 5 passed.

    A hint

    The bug is the len(items) - size + 1 part of the range call.

  2. Fix the discount and leap year bugs

    Using the same copy, fix discount.py so the percent is divided by 100, then fix leap.py so a year divisible by 100 is only a leap year if it is also divisible by 400. Run the tests after each fix. When we did this, the discount fix left 1 failure and the leap year fix brought it to 7 passed.

    A hint

    A leap year is divisible by 4, and either not divisible by 100 or divisible by 400.

  3. Add a test the assistant would need

    In test_leap.py add a test that 2100 is not a leap year and a test that 2023 is not a leap year. Before you run, predict which of them would fail against the original leap.py, which only checks year % 4 == 0. Then restore the original leap.py in your copy, run the tests, and compare with your prediction.

    A hint

    2100 is divisible by 4. 2023 is not.

  4. Predict the diff size

    In your copy, put the original leap.py back. Import changed_lines from assist.py in a Python prompt, read the old text, make a new text that changes only the return line to the full leap year rule, and print changed_lines(old, new). Then predict what the number becomes if the new text also drops the docstring line, and check it. We got 2 for the one line change and 3 with the docstring dropped.

    A hint

    A changed line counts as one removed line plus one added line. Importing assist needs the ollama package, which the video already installed.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

One small local model standing in for an assistant, three tiny planted bugs, one run. Not a comparison of any named tool. The run prints pass or fail and the diff size; the description of each edit comes from reading the diffs during preparation and the replies were not saved. The survey was run by Sonar, which sells verification tools, and the review guide is from crystl, which sells a coding terminal.

Up next in AI coding and productivity

More in AI coding and productivity

All AI coding and productivity videos

Related videos