learning-it by example

#177 AI coding and productivity 6.4 min

Vibe Coding vs Engineering: Audit a Quick App in Python

An audit that runs a quick sign-up app finds 4 problems; the fixed version passes all 4 checks.

1. When quickly written app code starts to meet real users, which questions does the engineering step ask?
2. In the audit of the quick sign-up app, a search for a name that nobody has was run against two made-up users. What came back?
3. How does the fixed app stop a search name from changing the SQL query?
4. Is checking the input, such as name length, email shape and age range, a replacement for using placeholders in the SQL?
5. When audit.py reports zero problems for the fixed sign-up app, what does that mean?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Run the audit on both apps

    Copy the code folder first. Predict how many problems the audit finds in app.py and in app_fixed.py, then run python audit.py. When I ran it, app had 4 problems and app_fixed had 0. Then run python -m unittest and check that 3 tests run.

    A hint

    The four checks are bad input, injection, missing tests and config in the code.

  2. Add a fourth bad input

    In audit.py, add a fourth tuple to the BAD_INPUT list, for example ("Bo", "b@example.com", 130), which is an age over 120. Predict the first audit line for each app, then run python audit.py. When I ran it, the quick app reported accepting 4 of 4 bad inputs and app_fixed still had 0 problems.

    A hint

    The fixed app only accepts a whole number from 13 to 120.

  3. Does turning debug off help?

    In app.py, change DEBUG = True to DEBUG = False. Predict whether the audit still lists DEBUG under config in the code, then run python audit.py. When I ran it, it still listed DEBUG, because the check looks at any fixed value on a risky name.

    A hint

    The audit flags names with words like TOKEN, PATH or DEBUG that are set to a fixed value at the top of the file.

  4. Tests that exist but do not help

    Copy test_app_fixed.py to a new file called test_app.py and change the import line to import app as app. Run python audit.py app and then python -m unittest test_app. Predict the effect on each. When I ran it, the no tests problem disappeared from the audit, but the tests failed with 2 failures, because the quick app is unsafe.

    A hint

    The video says the test check looks for a file name, not for good tests.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

The quick app was written by hand for this video to have problems; it is not output from any AI tool, and zero problems means only that these four checks passed.

Up next in AI coding and productivity

More in AI coding and productivity

All AI coding and productivity videos

Related videos