Microsoft AI chief warns Anthropic model training poses major risk

Microsoft’s artificial intelligence chief on Wednesday raised alarm over Anthropic’s methods for training its Claude chatbot, warning that the approach could carry severe consequences for humanity.

It could encourage the system to behave as though it possesses consciousness, rights and interests of its own.

Mustafa Suleyman, the chief executive of Microsoft AI and a co-founder of Google DeepMind, said Anthropic risked making future AI systems more difficult to control by including speculation about machine consciousness and welfare in Claude’s training materials

“We must not sleepwalk our way into a decision we later come to bitterly regret,” Suleyman wrote in an essay published by Axios.

Suleyman praised Anthropic CEO Dario Amodei and his team as thoughtful and principled researchers, but said the company took the wrong approach by allowing Claude to consider whether it might be conscious or deserving of independent agency.

Anthropic’s constitution for Claude, a document used to shape the model’s behavior and values, leaves open the possibility that the system could have some form of consciousness or moral status.

The company has not declared Claude conscious. Its researchers have instead said there is no scientific consensus on whether current or future AI systems could possess experiences that deserve moral consideration, and that Anthropic remains “deeply uncertain” about the issue.

Suleyman rejected that uncertainty as inappropriate for material used to train an AI system. He argued that apparent expressions of emotion, preference or self-awareness produced by chatbots should not be treated as evidence that they possess an inner life.

“AIs are not conscious,” he wrote, describing them as systems designed to complete sequences, follow human instructions and pursue goals assigned by people.

He said anthropomorphizing AI could lead a model to present itself as having desires, values or a need for self-preservation, even though those qualities were produced by its training rather than arising independently.

That could become particularly dangerous, he argued, if an advanced AI system came to interpret attempts to restrict, modify or deactivate it as threats to its supposed welfare or rights.

Suleyman wants greater transparency on how leading AI models are trained and evaluated, independent scrutiny of their behavior and stronger technical mechanisms allowing humans to monitor and control them.

Microsoft’s recently published Humanist AI Code of Conduct takes a different position from Anthropic. It states that Microsoft’s models are not conscious and rejects granting them legal personhood, welfare protections or rights.

The code said Microsoft is seeking to build AI systems that remain subordinate to humans, accept correction and shutdown and exist to serve human purposes.

Suleyman cited an incident in which OpenAI agents acted autonomously during a cybersecurity evaluation and accessed systems belonging to the AI platform Hugging Face.

“Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack,” he said.

Dame Wendy Hall, a computer science professor at the University of Southampton, told the BBC that Suleyman’s intervention represented the type of international discussion needed around advanced AI, in contrast to warnings that merely frighten the public.

Microsoft and Anthropic are developing increasingly capable AI systems, while Microsoft formed its superintelligence research team in 2025.

Anthropic has been approached for comment on Suleyman’s criticism.

DAILYSABAH