#98 AI security and governance 5 min
Audit Logs for AI Apps: Tamper-Evident Logging in Python
I edited one line of my own audit log. It noticed. Then I edited the last line.
Add a seventh request
Copy the code folder first. In compare.py add a seventh request to REQUESTS: ('dan', 'refund order D-1001, mail me at dan@example.org'). Run compare.py. Predict the number of lines and the users who used a write tool. You should see 24 lines, 7 of 7 requests with a start and an end, the write users ana, ben and dan, and 0 raw email addresses. Remove the new request again afterwards.
A hint
A refund request writes a start, a tool call, a tool result and an end line.
Edit a different middle line
In compare.py the middle edit changes line 9. Change it so that it edits line 2 instead: use middle = lines[:1] + [lines[1].replace('read', 'write')] + lines[2:]. Predict where the chain breaks, then run it. You should see the chain break at line 2, and the edit to the last line still not detected. Put the original middle line back afterwards.
A hint
The edited line is the one reported, because the line after it no longer matches.
Turn the masking off
In app.py change prompt=mask(prompt) to prompt=prompt in the start event, then run compare.py. Make sure REQUESTS has only the original six requests. Predict how many raw email addresses the log now has. You should see 2, and the log still intact. Change it back afterwards.
A hint
Two of the six prompts contain an email address.
Predict a deleted line
Think before you run anything. Suppose someone deletes line 4 from the middle of the log file, or the very last line. Predict what broken_at in audit.py reports in each case, using what the video says about the last line. In a copy of the folder you can test this by writing the file back without that line.
A hint
Ask which line would normally vouch for the deleted line.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
A pretend assistant with no model, six requests, one log file on one disk. The chain does not catch an edit to the last line, lines deleted from the end, or a rewrite of the whole file, unless the latest hash is kept somewhere else. No timestamps, so that the output repeats; one email pattern is not full masking of personal data; hashes are cut to 12 characters.
Up next in AI security and governance
More in AI security and governance
- #18 Prompt Injection Live: Attacking My Own LLM App
- #19 Redact Personal Data Before Your LLM Sees It (Python)
- #88 Indirect Prompt Injection: A Note That Gave Orders
- #89 Jailbreak-Test Your Own App: Eight Wrappers, Counted
- #90 Stop Leaking API Keys: Build a Secret Scanner in Python
- #91 Rate Limiting an LLM API: Token Bucket in FastAPI
Related videos
- #214 Build an AI Model Register and Risk Score Data governance
- #229 A Reference Architecture for Enterprise AI Agents, as Code Enterprise architecture and AI agents
- #230 Agent Gateway: One Front Door for Models, Tools, Agents Enterprise architecture and AI agents
- #231 MCP Gateway: Per-Tool Policy for Agent Tools (Python) Enterprise architecture and AI agents
