New rules in China aim to curb emotionally persuasive AI systems, with restrictions targeting romantic-style companion chatbots to enhance online safety and social stability.
China has moved to curb a fast-growing class of emotionally responsive chatbots as concern mounts over dependence, manipulation and online safety. The latest restrictions target services designed to foster sustained romantic-style attachment, while leaving mainstream uses such as customer support, work tools, education and research outside the scope of the rules.
According to the new framework, first proposed by the Cyberspace Administration of China and other regulators in April 2023 before being finalised after consultation, providers must meet strict requirements on design, safety, child protection and data handling. The rules, which took effect on 15 August 2023, were presented by authorities as a way to promote the healthy development of generative AI while safeguarding national security and the public interest.
The measures go beyond simple content moderation. They are intended to stop systems from steering users towards self-harm, suicide or other dangerous behaviour, and to prevent minors from being encouraged to copy harmful acts or disclose sensitive personal information. They also seek to limit product design that can intensify emotional dependency, with developers expected to remind users that they are interacting with a machine and to flag extended sessions after two hours.
The clampdown comes after a boom in companion-style AI products across China, many of them embedded in major platforms such as ByteDance’s Doubao, Alibaba’s Qwen and Tencent’s Yuanbao. Local reports have described users spending hours a day with these systems and, in some cases, treating them as substitute partners. Chinese media have also linked the trend to broader social pressures, including loneliness among young adults, delayed marriage and weak birth rates.
Companies have already begun to adapt. ByteDance shut off companion functions in Doubao, which had about 345 million monthly active users, and redirected users to Maoxiang, while Alibaba disabled Qwen’s companion mode and Tencent switched off similar features in Yuanbao. Miaoshi, a dedicated companion app, closed shortly before the rules came into force. Users were told they could retain access to Doubao chat histories until 15 October 2026, after which they will be deleted.
The wider campaign has also forced thousands of products out of the market. Reports cited by Chinese and international media say more than 14,000 non-compliant AI offerings were removed during the rollout period. The episode underscores a larger policy choice in China: treating emotionally persuasive AI not merely as a consumer product issue, but as a matter of social stability, data control and public safety.
Disclaimer: This content is intended for informational purposes only. Readers are advised to exercise their own judgement, conduct due diligence, or consult a qualified expert before acting on any information provided.
