Anthropic Updates Usage Policy: New Rules for High-Risk AI Applications

Anthropic has formally updated its Usage Policy, laying out a comprehensive framework that governs how its AI models, including the Claude family, can be used across consumer and enterprise applications. The policy, which takes effect immediately, categorizes restrictions into three tiers: Universal Usage Standards, High-Risk Use Case Requirements, and Additional Use Case Guidelines.

Universal Prohibitions

The Universal Usage Standards apply to all users and cover a wide range of banned activities. Prohibited uses include the creation or distribution of child sexual abuse material (CSAM), including AI-generated content; the development of weapons, explosives, or dangerous materials; and the design of tools for unauthorized access to systems or data, such as malware, ransomware, or botnets. The policy also forbids the generation of deceptive political content, synthetic media of political figures, or automated voter targeting and suppression campaigns.

Additionally, the policy outlaws the use of Anthropic models for impersonating humans in contexts where a user would be misled into believing they are interacting with a person. This includes presenting AI-generated outputs as human-generated without disclosure.

High-Risk Domains Require Oversight

A significant update is the introduction of mandated safeguards for what Anthropic calls High-Risk Use Cases, areas where AI-driven decisions can have direct impact on individuals or social equity. These include legal advice, medical diagnosis, insurance underwriting, financial advice, employment screening, housing eligibility, and academic testing. In each of these domains, Anthropic requires that a qualified human professional review all AI-generated advice or decisions before they are finalized or disseminated. Furthermore, the user must disclose to end consumers that AI is being used to assist in producing the advice or recommendation, at least at the start of each session.

“We believe that relevant human expertise should be integrated and that end-users should be aware when AI has been involved in producing outputs,” the company stated in the policy.

Guidelines for Chatbots and Agents

Anthropic also specifies that all consumer-facing chatbots and interactive AI agents must disclose to users that they are interacting with AI rather than a human. This disclosure must occur at the beginning of each chat session. The policy further addresses agentic use cases, noting that such applications must still comply with all prohibitions in the Usage Policy, with specific examples expected in a future Help Center article.

For products serving minors, Anthropic requires compliance with additional guidelines to be detailed soon, and any Model Context Protocol (MCP) servers listed in the company’s Connector Directory must adhere to its Directory Policy.

Enforcement and Reporting

Anthropic’s Safeguards Team will implement detection and monitoring systems to enforce the policy. Violations can result in throttling, suspension, or termination of access to Anthropic products and services. The company said it may also block or modify model outputs when inputs violate the policy. Users who believe model outputs are potentially inaccurate, biased, or harmful are encouraged to report issues via usersafety@anthropic.com or using in-product feedback mechanisms.

The policy notes that Anthropic may enter into tailored contractual agreements with certain governmental customers to accommodate their public mission if contractual restrictions and safeguards adequately mitigate harms.

Strategic Implications

The updated policy reflects a broader industry trend toward more structured governance of AI deployment, particularly in regulated industries. By requiring human oversight and disclosure in high-stakes applications, Anthropic is positioning Claude as a tool for augmentation rather than autonomous decision-making. The policy also signals a tightening of acceptable use in political and electoral contexts, consistent with growing regulatory and public scrutiny around AI-generated disinformation. The company continues to iterate on its safety frameworks as its models are deployed in increasingly sensitive domains.