learning-it by example

Back to the video

Pickle vs Safetensors: Why Loading a Model Can Run Code

Loading a pickle file, a format model files can use, ran code I had hidden inside it, four files out of four.

Video #99 ยท AI security and governance

Steps in the video

  1. 0:00 Loading a pickle file ran my code, four times out of four
  2. 0:30 The idea
  3. 1:03 Setup
  4. 1:32 Four booby trapped files
  5. 1:59 Two guards
  6. 2:24 Try them all
  7. 3:17 What this does not show
  8. 3:55 Mistakes to avoid
  9. 4:23 My take and next steps
  10. 4:51 Recap

Files

  • attacks.py (26 lines)
  • compare.py (53 lines)
  • guards.py (26 lines)
  • payload.py (6 lines)

Download the code (zip)

Tested with

Python 3.12.15 | safetensors==0.8.0, numpy==2.5.4

What this does not show

Four harmless payloads written by the author. The bad-module scan only reads the older pickle format used here (protocol 2); files saved with the default protocol show it no module names. A real model file may need more allow-list globals. Safetensors holds tensors, not everything in a model. No scanner service and no real model file were tested.

Sources

Learning IT by Example.