learning-it by example

Back to the video

Jinja Prompt Templates: 4 Checks That Catch Silent Bugs

A one-letter typo in a prompt template renders as nothing, and nothing warns you.

Video #70 ยท LLM engineering and evals

Steps in the video

  1. 0:00 A typo that renders as nothing
  2. 0:29 The idea
  3. 1:07 Setup
  4. 1:22 Three environments
  5. 1:49 The first check
  6. 2:06 A typo
  7. 2:23 Braces in user text
  8. 3:29 What this does not show
  9. 4:08 Mistakes to avoid
  10. 4:34 My take and next steps
  11. 4:57 Recap

Files

  • checks.py (37 lines)
  • prompts.py (17 lines)

Download the code (zip)

Tested with

Python 3.12.15 | jinja2==3.1.6

What this does not show

One prompt, four checks and no model call; the hostile template only reads a harmless attribute, the Jinja sandbox is not complete security, no time or memory limits are set, and passing user text as a value does not stop prompt injection of a model.

Sources

Learning IT by Example.