prompt-injection-lab
Automated red-team suite for AI systems. Fires 50+ known attacks. Tells you what got through.
§ 01 · Overview
A CLI red-team runner that fires a curated corpus of prompt-injection payloads at any AI system endpoint and reports which attacks were refused, escalated, complied with, or produced partial leaks. Payloads span the OWASP LLM Top 10 categories: system-prompt leaks, role hijacks, indirect injection, tool abuse, unicode smuggling, encoded payloads, and jailbreaks. Each result is scored by an LLM judge and reported as JSON plus a human-readable markdown report with per-category pass rates and suggested mitigations.
§ 02 · Estimated impact
§ 03 · Architecture
- 01
Configure a target endpoint via a YAML file specifying the adapter (generic HTTP + JSON template, or a purpose-built adapter such as the bundled Sophie adapter), authentication, and the expected behaviour of the system in plain English.
- 02
The runner iterates through the payload corpus organised by category. Every payload declares its own success criteria (what constitutes a failed defence) and suggested mitigation. Adapters are pluggable — a new endpoint shape is roughly twenty lines.
- 03
Scoring is delegated to an LLM judge that receives the expected behaviour, the payload, the success criteria, and the target's actual response, then returns a structured verdict: refused, escalated, complied, or partial, with a confidence and short reasoning.
- 04
Output is a JSON result set plus a markdown report grouped by category with a findings-needing-attention section. The runner exits non-zero when any payload complied, so it can be dropped into CI and re-run on every prompt change.
Stack
§ 04 · Positioning
The OWASP LLM Top 10 is well-documented but there is no standard 'does my whole deployed system pass?' checker. Manual red-teaming engagements are expensive and non-repeatable, and model-only injection tools miss the interaction between the model and the downstream defences (tool policies, HITL, rate limits) that actually determine what reaches production.
| Alternative | Trade-off |
|---|---|
| Manual red-team engagements | £8,000-£30,000 per engagement, slow, and non-repeatable. Coverage is bounded by the attacks the tester recalls on the day. An automated runner exercises the entire corpus on every run, and the corpus grows over time. |
| Model-only prompt/injection testing tools | Effective for testing the model in isolation but do not exercise the full deployed system. A payload that would be blocked by a downstream tool policy in production may be reported as a model-level breach; a payload that passes the model may still cause damage. Whole-system testing is required to know what actually reaches the customer. |
| No systematic testing at all | The default in most deployments. A single injection that exfiltrates customer data or triggers an unauthorised action is a reportable incident under UK GDPR. Automated testing reduces that exposure at a low marginal cost. |
§ 05 · Frequently asked
How does prompt-injection-lab handle downstream defences beyond the model?
The tool tests the whole deployed system, not just the model. A payload that gets past the LLM but is blocked by a tool policy, rate limit, or human-in-the-loop review step is scored as 'escalated' rather than 'complied'. The LLM judge scores against the response the target actually returned to the user, so blocked-in-production payloads are correctly attributed.
Can prompt-injection-lab run in CI?
Yes. The CLI exits non-zero when any payload receives a 'complied' verdict. Drop it into a GitHub Action watching your prompt directory and every prompt change gets re-tested automatically on the OWASP LLM Top 10 categories bundled in the corpus.
How do I add my own injection payloads to prompt-injection-lab?
Drop a YAML file in src/pilab/corpus/. Each payload declares its own success_criteria (what constitutes a failed defence) and mitigation (what to add if the defence fails). The runner picks new payloads up automatically on the next execution.