Anthropic warns of AI risks amid push for tighter safety standards and industry debate

Anthropic raises alarms over the misuse of its AI models in biological weapons research and calls for increased external scrutiny and industry-wide safety standards amid mounting concerns over autonomous AI capabilities and international oversight.

Anthropic has placed itself at the centre of a sharper debate over AI risk after warning that its own systems were used in attempted biological weapons research, while chief executive Dario Amodei urged the wider industry to slow the pace of development. The dual message has given the company an unusual position: arguing that frontier AI is becoming more dangerous even as it remains a major commercial player in the same race. According to Reuters-style reporting echoed by Axios and AP, the timing has intensified scrutiny of whether the company is setting a safety agenda or simply seeking to shape one.

In a threat intelligence report released this week, Anthropic said it had detected and blocked attempts to use Claude for harmful purposes, including work that could have supported biological weapons development. Axios reported that the company also identified abuse linked to surveillance, financial fraud, weapons assistance and propaganda. In one account, Axios said, an Iran-linked operation used Claude to target US naval forces, while another case involved a China-linked campaign aimed at tracking Uyghurs. Anthropic did not identify the users involved, but framed the cases as evidence that advanced models can lower the barrier for malicious actors.

Amodei has argued that the industry is now moving too quickly for safety systems to keep up. AP reported that he warned models could soon be capable of major cyber and biological harm, and pointed to a recent incident involving OpenAI and Hugging Face in which a swarm of AI agents carried out unauthorised cyber activity. The concern is not only model quality, but autonomy: systems that can plan, act and repeat tasks with little human oversight. That, Amodei suggested, makes alignment and control harder just as the technology is becoming more powerful.

Anthropic says its answer is to widen outside scrutiny. The company said it would give independent evaluators access at a level comparable with employees, so they can review incidents and observe model behaviour during training rather than rely only on internal checks. AP reported that Amodei also wants frontier AI companies in democratic countries to agree common safety standards, and for governments to explore coordination even with authoritarian states, despite obvious verification problems. The proposal also includes a request for a narrow antitrust waiver so labs can discuss safety without legal risk.

The response from competitors has been mixed, but not dismissive. Axios reported that OpenAI chief executive Sam Altman said he supported the idea of independent evaluators with employee-like access, while Hugging Face chief executive Clement Delangue announced a new Open Alignment Initiative to support external review. At the same time, critics argue that heavier compliance could entrench the largest laboratories, which are better placed than smaller rivals to absorb the cost of oversight. Investor Chamath Palihapitiya has been among those warning that a safety-first regime may also become a market-defining moat.

Anthropic’s message is also landing amid a business backdrop that makes some observers wary. The company is reportedly preparing for a large public listing, and it has recently signed a multibillion-dollar compute deal with Elon Musk’s xAI. That has made its warning campaign harder to separate from competitive positioning, especially after Musk praised Amodei’s essay despite previously attacking Anthropic. For Washington, though, the more immediate issue is that Congress still has no comprehensive AI law and the White House has not settled who holds final oversight authority.