#169 AI coding and productivity 6.4 min
Write Coding-Assistant Rules That Work
Two of my seven rules could not be checked; a script checks the other five.
Break the after file
Copy the code folder first. In after.py delete the docstring line inside apply_discount. Predict which rule fails, then run python run.py. On my machine the after line said 4 of 5 rules pass, failed R3.
A hint
The docstring check asks Python for each function's docstring.
Find the gap in the key rule
In after.py change the default in os.environ.get("SHOP_KEY", "") to "abc123". Predict whether R4 still passes, then run python run.py. Next replace the whole line with SHOP_KEY = "abc123", then with SHOP_TOKEN = "abc123", and run each time. On my machine the first and third versions passed 5 of 5 and the second failed R4.
A hint
The key check looks for the capital letters KEY followed by an equals sign and a quote.
Add a sixth concrete rule
Add the line - R6 Function names are lower case to the concrete section of RULES.md. In checks.py write r6 with the same two arguments as the other checks, using ast.walk to find every FunctionDef and checking that name == name.lower(), and add it to CHECKS. Run python run.py. On my machine before.py then passed 1 of 6 and after.py passed 6 of 6.
A hint
Copy the shape of r3_docstrings, which also collects FunctionDef nodes.
Turn a vague rule into a check
Think of a vague rule you might give an assistant, such as keep functions short. Rewrite it as a concrete rule with a number, add it to RULES.md as R7, and write a check for it in checks.py. Then run python run.py on before.py and after.py and see which files pass.
A hint
Each check must answer pass or fail from the code text and the parsed tree. A line count per function needs end_lineno on each FunctionDef.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
The before and after files are hand-written examples, not output from any assistant; no assistant was run.
Up next in AI coding and productivity
More in AI coding and productivity
- #20 AI Coding Assistant Workflow: Copy, Test, Read the Diff
- #172 Refactor Safely With AI: Characterisation Tests (pytest)
- #173 Debug with AI: Give It a Traceback and a Minimal Repro
- #174 Review AI-Generated Code Fast: Checklist vs Bandit vs Tests
- #177 Vibe Coding vs Engineering: Audit a Quick App in Python
Related videos
- #05 Structured Outputs in Python: Valid JSON Is Not Correct JSON LLM engineering and evals
- #56 Prompt Versioning: Compare Changes Case by Case LLM engineering and evals
- #63 Few-Shot Examples: How Many Is Enough? A Measured Test LLM engineering and evals
- #64 Chain of Thought: When Step by Step Helps, and Its Cost LLM engineering and evals
