Cybercriminals exploit AI account theft as new attack surface emerges at Black Hat

Security experts warn that stolen and traded AI accounts are creating a stealthy new front for cyberattacks, with criminals capitalising on the growing use of generative AI tools within organisations.

At Black Hat in Las Vegas, security specialists warned that corporate use of generative AI has created a fresh attack surface inside organisations. The concern is not only that employees are using approved AI services, but that those accounts are now being stolen, traded and abused in ways that can be harder to detect than traditional intrusions. According to a report by Axios, credentials for tools such as ChatGPT, Claude and Gemini are increasingly being bought and sold on criminal markets through infostealer malware.

Adam Meyers, senior vice-president of counter-adversary operations at CrowdStrike, said this pattern has been growing since ChatGPT took off in late 2022. In practice, attackers who gain access to employee AI accounts can shift their own computing costs on to the victim company while also blending malicious activity into legitimate usage. That makes detection more difficult for security teams, especially when the account owner may normally use the service at irregular hours.

CrowdStrike has previously warned that identity-based attacks are rising sharply across the wider threat landscape. In its 2023 threat hunting report, the company said Kerberoasting attacks increased 583%, while adversaries using legitimate remote monitoring and management tools rose 312% year on year. Its 2023 global threat report also described a 95% rise in cloud exploitation and a 112% increase in access broker adverts on the dark web, underscoring how quickly attackers are moving towards stealthier, identity-driven methods. Independent research has also pointed to AI being used to automate attack-surface generation and to support coordinated bot activity online.

Dave Gray, chief executive of Bugcrowd, told the conference that this is still an early-stage threat but one that could become central to cyberattacks. He argued that organisations are rapidly adopting sanctioned AI tools while also losing visibility over unauthorised systems quietly entering the network. Gray cited cases in which his company warned staff not to download an open-source AI agent called OpenClaw, only to find workers still trying to install it. Dana Simberkoff, chief risk, privacy and information security officer at AvePoint, said tools for verifying AI agent identity are still evolving, leaving a market opportunity for better controls as well as a security gap that criminals can exploit.