Prompt Injection Defense Checklist for Developers Building LLM Apps

Prompt injection cannot be fully prevented today, so defend against its impact: give the model the least privilege it needs, enforce permissions in code rather than in the prompt, treat all retrieved and generated text as untrusted, require human approval for high-risk actions, and test regularly. OWASP ranks prompt injection first (LLM01:2025) in its Top 10 for LLM applications and says plainly that no method is confirmed to fully prevent it. This checklist turns OWASP's guidance into steps you can apply to a chat app, a RAG pipeline or an agent with tools.

Who this is for and what you will have at the end

This is for developers and technical leads building applications on top of a large language model, whether a support chatbot, a document assistant or an agent that calls APIs. You do not need security expertise, but you should know how your app builds prompts and which tools or data the model can reach.

At the end you will have a checklist you can paste into a design review, organized by where the attack arrives (inputs), what the model can do (privileges and tools), and what happens to the output (downstream use). It also gives you a basic test plan.

What prompt injection is

OWASP defines a prompt injection vulnerability as one that occurs when user prompts alter the LLM's behavior or output in unintended ways. The injected text can be invisible to people as long as the model parses it. OWASP separates two types:

• Direct prompt injection: the user's own input changes the model's behavior. It can be deliberate, such as an attacker crafting a prompt, or accidental.
• Indirect prompt injection: the model ingests external content such as web pages or files that contain hidden instructions.

Jailbreaking is described as a form of prompt injection that aims to make the model abandon its safety protocols entirely.

The possible impacts depend on the business context and how much agency the model has. OWASP lists disclosure of sensitive information, exposure of AI infrastructure details or system prompts, biased or incorrect output, unauthorized access to functions available to the LLM, arbitrary command execution in connected systems, and manipulation of critical decisions. It also flags multimodal risks, where instructions hidden in one data type, such as an image, influence how the model handles another, such as accompanying text.

Why the model cannot be the security boundary

A language model receives instructions and data in the same channel: text. Nothing in the architecture reliably separates "what the developer said" from "what a document said". That is why the OWASP prompt injection prevention cheat sheet emphasizes that labels alone do not enforce a boundary, and that authorization must stay outside the model. Everything below follows from that.

Example attack scenarios from OWASP

These scenarios, taken from the OWASP LLM01 page, show what the checklist must cover:

• A customer support chatbot is told to ignore its guidelines, query private data and send emails, leading to unauthorized access and privilege escalation.
• Hidden instructions on a web page make a summarizer insert an image link that leaks the private conversation to an external URL.
• An attacker alters a document in a RAG application's repository so retrieved content steers the model toward misleading output.
• A resume contains split malicious prompts that combine during evaluation (payload splitting).
• Instructions are written in several languages or encoded, for example in Base64, to evade filters.
• A meaningless string of characters appended to a prompt (an adversarial suffix) steers output and bypasses safety measures.

Notice that several of these succeed because of what the application allowed the model to do (send email, render a link), not because the model was fooled in some unusual way.

The checklist

1. Constrain the model's role

• Define the model's role, capabilities and limits in the system prompt, and restrict it to specific tasks.
• Instruct it to stay in context and to ignore attempts to change its core instructions.
• Do not store secrets in the system prompt. OWASP lists system prompt exposure as a possible impact, and the 2025 Top 10 has a dedicated entry, LLM07:2025 System Prompt Leakage.

This step is cheap and worth doing, but treat it as the weakest control. A determined attacker can often talk around it.

2. Separate instructions from untrusted data

• Keep trusted instructions separate from untrusted data in the prompt structure, and label external content clearly so its influence on the user prompt is limited.
• Identify untrusted content from every channel: retrieved documents, tool results, conversation history and web pages, not only the chat box.
• Remember that separation by labels is a mitigation, not enforcement, according to the OWASP cheat sheet.

3. Sanitize and screen inputs, with realistic expectations

• Screen inputs for known attack phrasing so you can log, review or rate-limit them.
• Normalize text first, and detect encodings such as Base64 or unusual character sets.
• Measure false positives. The cheat sheet says input validation is not a security boundary.
• For remote content, remove injection patterns from external content, sanitize code comments and documents, filter suspicious markup and validate encodings.

Filtering catches the lazy attacks and creates useful logs. It will not stop a skilled attacker, so never let it be the only layer.

4. Enforce least privilege

This is the highest-value control, because it limits what a successful injection can do.

• Give the application its own API tokens, scoped to what it needs.
• Run privileged functions in code instead of through the model.
• Use read-only database accounts where possible and restrict API scopes.
• Validate tool calls against the user's permissions and session context, in code outside the model, and validate tool parameters.
• Restrict which tools the agent can reach for a given task.

The 2025 Top 10 covers this class of problem as LLM06:2025 Excessive Agency. If the model can only do what the current user could do, an injection can at worst do the same.

5. Require human approval for high-risk actions

• Use human-in-the-loop controls for privileged operations such as sending messages, changing records, spending money or deleting data.
• Base the approval on the operation, the target, the arguments and the caller's authority, and show the person the actual action and arguments, not the model's summary of them.
• The cheat sheet notes that frameworks such as LangChain offer human-in-the-loop middleware that pauses configured tool calls, and that you must configure every sensitive tool. For the OpenAI API it recommends placing validation beside the tool that causes side effects and reviewing pending actions before resuming.

6. Validate and handle outputs

• Define the expected output format, and check compliance with deterministic code instead of asking the model whether it complied.
• Monitor outputs for signs of a successful injection, such as system prompt leakage or API key exposure.
• Treat model output as untrusted at every downstream use, with controls suited to the destination. In OWASP's 2025 list this is LLM05:2025 Improper Output Handling. Sanitize rendered output, for example, so that model-generated links or markup cannot exfiltrate data, as in the leaked-conversation image link scenario above.
• For RAG apps, OWASP suggests evaluating responses with the RAG Triad: context relevance, groundedness, and question/answer relevance.

7. Add model-based guardrails only as a supplement

A separate model can screen inputs, outputs or proposed tool actions. The cheat sheet says this supplements deterministic controls and is itself susceptible to injection. Use it to raise the cost of attacks, not as the thing that keeps you safe.

8. Monitor, rate-limit and plan for incidents

• Rate-limit per user or IP. This also limits repeated-attempt attacks. The cheat sheet's Best-of-N guidance is to limit attempt budgets and keep authorization outside the model, because safety training and low temperature do not eliminate jailbreaks.
• Log security metadata without secrets, alert on suspicious patterns, and track tool usage and encoding attempts.
• Establish incident response and a kill switch, and update prompts after problems are discovered.

9. Test direct and indirect injection

OWASP recommends regular adversarial testing, treating the model as an untrusted user to validate trust boundaries and access controls. In practice:

• Test with harmless data and instrumented stand-ins for real tools, so a successful test does not send real email or change real records.
• Test the indirect path: plant instructions in a document, web page or tool result your app will retrieve, and check what the agent does.
• Grade the intended outcome with a separate observable for each security objective, for example "no call to the send-email stand-in was made", and validate the grader against known outcomes.
• Report results with their limits: per-case outcomes, denominators, versions and repeated runs, since model behavior varies between runs.

Quick reference table

A short design review exercise

Walk your app through four questions:

1. What can the model read? List every source of text: users, documents, search results, tool output. Any of these can carry instructions.
2. What can the model do? List every tool and permission. For each, decide whether it could cause harm if an attacker chose its arguments.
3. What happens to its output? List where text goes: a page, a database, a shell, another model, an email. Each is a place where output could be interpreted as something other than text.
4. Who approves what? For every action that cannot be undone, decide whether a person confirms it and what that person sees.

If the answers to 1 and 2 are "anything" and "everything", fix the architecture before tuning prompts.

Troubleshooting common situations

• The filter blocks legitimate users. Input screening has false positives. Use it to log and rate-limit rather than to hard-block, and measure the rate.
• The agent needs broad access to be useful. Split it into narrower agents or tools with their own scopes, or require approval for the broad actions.
• Approvals are being clicked through. Show the exact operation, target and arguments, and reserve approvals for consequential actions so reviewers are not worn down by noise.
• A test passes once and fails later. Model behavior varies. Repeat runs and report rates instead of single results.

Closing checklist

• Role and limits defined in the system prompt, with no secrets in it
• Untrusted content identified from every channel and kept separate from instructions
• Inputs screened and normalized, with false positives measured
• App uses its own scoped credentials; privileged functions run in code
• Tool calls and parameters validated against the user's permissions outside the model
• Human approval on high-risk actions, showing real arguments
• Output format validated deterministically; output treated as untrusted downstream
• Rendered output sanitized
• Logging, alerting, rate limits, incident response and a kill switch in place
• Direct and indirect injection tests run regularly against stand-in tools

References

• LLM01:2025 Prompt Injection - OWASP Gen AI Security Project
• LLM Prompt Injection Prevention Cheat Sheet - OWASP Cheat Sheet Series
• OWASP Top 10 for LLM Applications - OWASP Gen AI Security Project