Skip to main content
SkillTycoonSign in
~55 min

Course

The Prompting Playbook: Debug a Production Prompt

7 lessons

Start free — no account, no card.

One whole course is free. After that, one payment of $10 opens all 20 courses in the catalogue today. What’s included →

Your production prompt worked, then you moved to a new model and the test cases started failing. This is the repair method, in order. Build a small eval suite of control, edge and capability-boundary cases, so you can tell a real improvement from a feeling. Strip the pasted-website cruft and the untrue persona. Carve the prompt into tagged sections so role, policy, guidelines and tone stop competing. Add an output contract and enforce it in the harness with a stop sequence. Then isolate one failing case at a time and find the instruction that made the wrong answer the compliant one. Worked end to end on a messy customer-support prompt for a fictional telco, then run on a prompt of your own. Based on “The Prompting Playbook” by Margot van Laar, Code with Claude.

What you’ll learn

  1. 1.Measure before you change anything

    A production prompt regresses on a new model. Before touching a word of it, work out what kind of problem you have — and build the eval suite that lets you tell whether any change actually helped.

    2 lessons

    1. Why a prompt that worked stops working
    2. The eval suite comes before the fix

  2. 2.Clean it before you fix it

    General hygiene first, specific failures second. Strip the pasted website furniture and the untrue persona, carve the remaining wall of text into tagged sections, then pin the answer down with an output contract — and watch the eval move without a single new instruction.

    3 lessons

    1. Cruft: what the prompt should never have contained
    2. Structure: role, policy, guidelines, tone
    3. Output contracts and stop sequences

  3. 3.Chase one failure at a time

    Two cases pass, three failure modes remain. Take exactly one: read what the model actually said, then find the instruction that made that answer the compliant one and rewrite it so the conflict is resolved. Then run the whole method on a prompt of your own.

    2 lessons

    1. Output contracts and stop sequences
    2. Find the instruction that caused the failure
    3. Run the method on a prompt of your own

How this course works

This is an interactive course, not a video playlist. Lessons mix short explanations with hands-on activities — quizzes, drag-and-drop exercises, branching scenarios and step-ordering challenges — and you have to complete each activity before moving on, so you practise rather than skim. Your progress is saved as you go, so you can stop after any lesson and pick up where you left off.

Topics covered

  • prompt-engineering
  • evals
  • llm
  • production
  • claude
  • anthropic
  • debugging
  • interactive