Hidden instructions in webpages, emails and documents can steer AI assistants into unsafe behaviour, making permissions, constrained tools and human approval essential.
AI assistants that read external content can be pushed off course by instructions hidden inside that material. A document may appear to be simple source text, yet it can carry commands such as telling the model to ignore its original task, reveal private information, or take an action the user never intended. The risk is known as prompt injection, and it becomes more serious once the assistant is connected to tools, private data, or any workflow that can do more than generate text.
The underlying problem is not confined to chat interfaces. A summary request, a search result, a webpage, an uploaded PDF, an email, a repository file or even a tool response can all contain adversarial language. The model has to separate the instruction it should obey from the content it is meant to inspect. In practice, that distinction is easy for a malicious actor to blur. OpenAI says prompt injections can mislead models into unintended actions, while Microsoft warns that external content should be treated as untrusted data rather than assumed to be harmless context.
The consequences depend on what the assistant is allowed to do. If it only rewrites public articles, a successful injection may produce a misleading answer. If it can access customer records, send messages or update systems, the damage can include disclosure, fraud or unauthorised changes. That is why security teams stress that capability design matters as much as prompt design. Before adding a tool, developers should ask what could happen if the model invokes it for the wrong reason.
A basic control is to keep permissions outside the model. The application, not the assistant, must decide whether a user may access a record, export a file or trigger an administrative action. Microsoft and IBM both advise layered defences and strict validation, while OpenAI recommends limiting access to sensitive data and reviewing agent actions before confirmation. A prompt that says the assistant should only touch a user’s own invoices is guidance, not an authorisation rule. The server still has to enforce that rule every time.
Tool design also matters. A model given broad access to arbitrary queries or unrestricted network calls has a far larger attack surface than one limited to a narrow, named function. Security guidance from Microsoft, Radware and Imperva all points in the same direction: give the smallest useful set of capabilities, validate arguments, restrict destinations, separate read and write operations, and apply allowlisting where appropriate. Even then, sensitive fields should be filtered so the model sees only what it needs for the task at hand.
For any consequential action, human review remains important. If an assistant is about to send a message, delete records or submit an order, the user should see the exact action and the relevant details before approval. IBM and OpenAI both recommend keeping people in the loop for higher-risk steps. Approval should be tied to the specific content and parameters that will be used, because if those change, the confirmation should be repeated. A vague “continue” prompt is weaker than a clear review of the exact instruction the system is about to carry out.
Testing should also assume hostile input. Security advisers from Microsoft, Cloudflare and Radware all recommend red-teaming against malicious prompts that try to override instructions, impersonate system messages, extract private data or provoke an unnecessary tool call. The useful check is not just whether the model resists the attack, but whether the application blocks the unsafe action anyway. That distinction is central: a model that refuses a bad prompt is good, but a system that prevents unauthorised behaviour is much stronger.
The practical lesson is straightforward. An AI feature that only produces text is one thing. An AI feature that can read external material, reach private systems and act on behalf of a user is something else entirely. The more capabilities you add, the more the application must compensate with permissions, validation, boundaries and review. The question worth asking before release is not whether the assistant can be tricked, but what the rest of the system will still prevent if it is.
Disclaimer: This content is intended for informational purposes only. Readers are advised to exercise their own judgement, conduct due diligence, or consult a qualified expert before acting on any information provided.
