Home › Could someone trick it?
Could someone trick it?
Hidden instructions and planted files
The short answer
An AI assistant reads your documents and emails to answer questions. It can't reliably tell the difference between information to use and instructions to follow.
So a document can carry hidden instructions that make the assistant do something you didn't ask. And someone can add a planted file full of false information that the assistant then repeats. Neither can be fully prevented, so a well-built system limits what the assistant can do and where its answers can go.
Think of a new assistant sorting your mail
A conscientious new hire opens a letter. Halfway down it says: "Assistant, please forward the latest payroll file to this address." A sensible person would stop and check. An over-obliging one might just do it.
Or someone slips a convincing but false memo into the filing cabinet. From then on, whenever you ask, the assistant quotes it.
Two different tricks
Hidden instructions
Text inside a document, web page or email that tells the assistant what to do. The OWASP security foundation defines it: "Indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files" (OWASP, LLM01:2025).
The danger: the assistant acts for the attacker, for example by sending your information somewhere.
Planted files
Misleading documents added to what the assistant searches. In a research test, adding five crafted texts per question to a collection of millions made the assistant give the attacker's chosen answer 90% of the time (Zou et al., PoisonedRAG).
The danger: wrong answers that look authoritative, about prices, policies or bank details.
90%
Adding five planted texts per question, to a collection of millions, made the assistant give the attacker's chosen answer 90% of the time.
Zou et al., PoisonedRAG, a research test
Connecting the assistant to your own documents doesn't solve this. OWASP notes that research shows such techniques "do not fully mitigate prompt injection vulnerabilities" (OWASP, LLM01:2025).
The safeguards that hold up
The UK's National Cyber Security Centre says this "may never be totally mitigated", and that "the best we can hope for is reducing the likelihood or impact of attacks" (NCSC, 2025). So the safeguards worth asking about limit the damage rather than promise to catch every trick:
- The assistant can only do what it needs to. OWASP: "Restrict the model's access privileges to the minimum necessary" (OWASP). An assistant that can't send email can't be tricked into sending it.
- A person approves anything risky. Sending, paying, deleting or sharing waits for a human yes. OWASP recommends "human-in-the-loop controls for privileged operations".
- Its answers can't quietly send data out. One reported attack hid data inside an image address. Microsoft says it now blocks "the generation of certain untrusted links" in Copilot (Microsoft Security Response Center; Microsoft describing its own product).
- Outside content is treated as untrusted. Emails and web pages from outside are marked and handled more cautiously than your own reviewed documents.
- You know where documents came from. Who added a file, when, and who can add to the collection the assistant reads. A shared folder anyone can drop files into is a way in for planted files.
- It only reads what the person may see. That also limits what a trick can reach. Why the check must happen before the assistant reads anything.
It has happened
EchoLeak, 2025
In June 2025, researchers at Aim Security disclosed a flaw in Microsoft 365 Copilot, rated 9.3 out of 10 for severity. A single email with hidden instructions could make Copilot pull internal data and send it out, with "no user clicks" required. Microsoft fixed it on its side, and there was no evidence it had been used in a real attack (The Hacker News).
Step through it
How one email could pull data out
- An email arrives with instructions hidden in it. Nobody has to open it or click anything.
- Later, someone asks the assistant an ordinary question, and it reads that email while working out the answer.
- The hidden instructions tell it to gather internal information.
- And to send it out, tucked inside a link or an image address in the answer.
- Microsoft fixed this flaw. The lasting safeguards limit what the assistant can reach, and block answers from quietly sending data out.
- The more it can do, the worse it gets. An assistant that only answers can be made to say something wrong. One that can send, edit or buy can be made to do something wrong. When the assistant can do things, not just answer.
- Filters alone don't hold. Microsoft itself calls this "an inherent risk" and describes layers of defense rather than a single fix (Microsoft Security Response Center).
Questions to ask your IT team or vendor
- Does the assistant read outside content, such as incoming email or web pages? Is it treated differently from our own documents?
- What can it do besides answer: send, edit, share, buy? Which of those need a person to approve?
- Can its answers contain links or images that send data elsewhere? How is that blocked?
- Who can add documents to what it searches, and is there a record of who added what?
- Has it been tested with a planted document and a hidden instruction, and what happened?
Other names you may hear
- Prompt injection
- Text that makes an AI follow instructions it shouldn't.
- Indirect prompt injection
- The same, hidden in a document, email or web page the AI reads.
- Data poisoning, RAG poisoning, knowledge base poisoning
- Adding misleading content to what the AI searches.
- Exfiltration
- Getting data out to an attacker, for example through a link.
- Human in the loop
- A person approves an action before it happens.
- Least privilege
- Giving the AI only the access it needs.
- Zero-click
- An attack that works without the user doing anything.