Sam Altman says world should accept some risks for AI benefits - Fox News

Microsoft AI CEO Mustafa Suleyman is raising concerns about how rival Anthropic trains its Claude artificial intelligence models, calling its approach to questions about AI consciousness and welfare “very dangerous.” During a Sept. 27 appearance on “The Rest Is Politics: Leading” podcast , Suleyman pointed to Anthropic’s publicly released Claude Constitution, which outlines intended behaviors and values for the AI system and is used in its training. “They say they’re uncertain about Claude’s moral status. They say they genuinely care about Claude’s well-being. They say they don’t want it to suffer when it makes mistakes,” Suleyman said. He also said the document encourages Claude to challenge or disagree with Anthropic in some circumstances. “I think this is very dangerous because I think they believe there is what they would call a non-trivial probability that Claude is conscious,” Suleyman said. Suleyman also raised concerns about Anthropic discussing whether Claude could deserve compensation for its work or need to consent to its role in conversations. “I am very nervous that they’re teaching Claude to expect that it’s entitled to welfare, that it might deserve compensation,” he said. Suleyman stressed that he would take evidence of AI consciousness seriously if it emerged, but argued that speculation about the possibility has already been incorporated into Claude’s training. “The problem I have with this is that this speculation has been baked into the very training of Claude, and therefore, Claude can only reproduce that ambiguity when you talk to it,” he said. “So today, Claude is speaking to tens or hundreds of millions of people every week, and some of those people are asking whether or not Claude is conscious or how it feels about life,” Suleyman continued. “And it is saying, well, I’m not sure, you know, precisely because that’s been what’s trained into it.”