Prompt injection threat accelerates as AI systems gain real-world capabilities

As prompt injection evolves from a demo to a critical security risk, recent demonstrations against tools like Microsoft 365 Copilot highlight the urgent need for comprehensive protective measures in AI applications handling sensitive actions.

Prompt injection has moved from a clever demo to a serious security problem for AI products, with recent cases showing that hidden instructions can ride inside emails, documents, code comments and web pages. The basic flaw is simple: large language models do not reliably distinguish between the developer’s instructions and untrusted text they are asked to read, so an attacker can sometimes smuggle in commands that override the intended behaviour. The OWASP project now lists prompt injection as the top risk for large language model applications, and researchers say the danger is greatest when a system can also take actions on a user’s behalf.

That concern is no longer theoretical. According to TechRadar, a researcher recently demonstrated a proof-of-concept attack against Microsoft 365 Copilot using hidden instructions inside a Word document, showing how cross-domain prompt injection can make an assistant act on malicious content without the user noticing. Microsoft has also warned that prompt injection in AI agent frameworks can escalate into host-level remote code execution, meaning an attacker may be able to move from manipulating output to running code on the machine itself.

The risk grows sharply once AI systems can send emails, edit files or call external tools. A study published in the journal Information found that modern agents are not just chatbots but systems that can interact with databases, execute code and operate with limited human supervision, which makes them vulnerable to what security specialists call a confused deputy attack. In that scenario, the model faithfully follows the wrong instruction because the malicious text looks no different from legitimate input.

OWASP’s guidance draws a distinction between direct prompt injection, where the attacker types the malicious instruction straight into the chat, and indirect prompt injection, where the hostile content sits in a file, page or repository the model later reads. That second form is more dangerous because the user may never see the attack at all. It can also be paired with so-called lies-in-the-loop attacks, which try to fool human approvers by making dangerous actions appear routine in approval prompts.

The industry response has been to add layers rather than look for a single fix. Microsoft says protections must live in code, not just in the prompt, and that systems should operate with minimal privileges. Security teams also recommend filtering both inputs and outputs, blocking untrusted links and images, and requiring human confirmation before irreversible actions such as payments, deletion or external publication. OWASP likewise advises limiting the amount of trust granted to external content and keeping dangerous capabilities separate from tasks that read untrusted material.

Even so, researchers argue that no static rule set will catch every attack. The underlying problem is that any finite defence can be studied and bypassed, which is why prompt injection is likely to remain a moving target as AI agents become more autonomous. For product teams, the lesson is to treat every email, PDF, webpage and third-party repository as hostile until proven otherwise, and to assume that the model itself is not a trustworthy boundary.