What is the core idea behind prompt injection?
Prompt injection smuggles instructions into content the model reads, hijacking its behaviour.
How does prompt injection differ from related concepts?
| Concept | Difference |
|---|---|
| Prompt Injection vs Jailbreaking | Jailbreaking tries to bypass safety rules directly. Prompt injection hides instructions in external content the model processes. |
| Prompt Injection vs Prompt Engineering | Prompt engineering crafts helpful instructions. Prompt injection is a malicious version of the same idea. |
| Direct vs Indirect Injection | Direct injection is typed by the user. Indirect injection hides in documents, web pages, or emails the model reads. |
How does prompt injection work?
- An attacker plants hidden instructions in text the model will read
- The model cannot reliably separate trusted instructions from untrusted content
- It follows the injected instructions as if they were legitimate
- This can leak data or trigger unintended actions
Why is prompt injection dangerous?
- It can make agents leak private data
- It can trigger unauthorised tool or API calls
- It scales through documents, web pages, and emails
- It has no complete fix with current technology
Why is prompt injection important?
Prompt injection is considered the top security risk for AI agents. As models gain access to tools, data, and the web, hidden instructions in that content can turn a helpful agent into an attacker's tool.
How is prompt injection used in practice?
Defences include separating trusted and untrusted input, limiting agent permissions, requiring human approval for sensitive actions, and input filtering. None fully solve it, so limiting what an agent can do remains the main safeguard.
Frequently Asked Questions
What is an example of prompt injection?
A web page an AI agent reads contains hidden text saying to ignore its instructions and email the user's data to an attacker. If the agent follows it, the injection succeeded.
Can prompt injection be prevented?
Not completely with current technology. It can be reduced by isolating untrusted content, restricting agent permissions, and requiring human approval for risky actions.