Indirect Prompt Injection

Indirect Prompt Injection occurs when a model retrieves reference material (e.g., from a third-party site, document, or tool output) that has been maliciously altered to include harmful instructions. These embedded prompts override or interfere with the intended task, causing the LLM to misbehave, often in subtle and unexpected ways. Input/Output Checks and Prompt-based Mitigations will reduce the likelihood of the risk, but may not eliminate it. In sensitive contexts, the model should be treated as an untrusted user and given a minimum set of permissions (see the Excessive Agency risk for further discussion). If the output is presented to an individual, they should be aware of its potentially untrustworthy content.

This risk is relevant in systems that use Retrieval-Augmented Generation (RAG), call external APIs, or embed LLMs into workflows that pull from user-editable fields (e.g., emails, documents, calendars, or websites). It poses a unique challenge because the user accessing the system may not be the attacker—the malicious payload originates in external content.

- LLMail-Inject Challenge: A proof-of-concept attack inserted a payload into a Google Calendar entry, which was then interpreted by an LLM-integrated assistant, bypassing normal user prompts. This scenario is explored in Microsoft's LLMail-Inject challenge, which evaluates prompt injection defenses in LLM-integrated email clients.