Microsoft CEO says superintelligence must remain 'under human control' - Fox Business

close video Microsoft stock target raised to 550 as AI potential grows CFRA Research Senior Vice President Angelo Zino joins 'Making Money' to discuss Microsoft stock as CFRA raises its price target to 550. Microsoft CEO Satya Nadella said Sunday that the pursuit of superintelligence — AI technology that could surpass humans across virtually every cognitive task — should focus on "helping humanity" and remain under "human control." In a post on X, Nadella called for broader AI adoption through what he described as a "frontier ecosystem" where both closed- and open-source AI models can "thrive" across countries, communities and businesses. "Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it's not worth pursuing," Nadella wrote. "We also need to accelerate and spread the benefits of AI, such that they are diffused broadly across countries, communities, and companies. This requires a frontier ecosystem in which both closed and open-source models can thrive." BILL GATES OUTLINES THE STAKES OF THE AI ERA: 'GREATEST EQUALIZER... OR WORST SOURCE OF INJUSTICE' Microsoft CEO Satya Nadella said Sunday that the pursuit of superintelligence must remain focused on "helping humanity" while staying under "human control." (Fabrice COFFRINI / AFP via Getty Images) He added that organizations should be able to build AI systems using their own data rather than becoming dependent on a single model provider. " For firms , it’s imperative that they retain full control over their unique and tacit knowledge," Nadella wrote. "Every organization should be able to build its own continuous learning loop/hill climbing machine, without becoming dependent on any one model provider, and have the ability to embed its own knowledge into models and weights they control." Nadella called for a deliberate approach to AI development, saying Microsoft supports concepts such as "embedded evaluators." "So, in this context, we welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal," he said. "We also welcome ideas like ‘embedded evaluators’ and the broader efforts to develop the mechanisms to make this more than just talk." NVIDIA, MICROSOFT URGE US TO AVOID BROAD RESTRICTIONS ON OPEN AI MODELS Nadella also said advanced AI development should not be controlled by a handful of companies. (Craig T Fruchtman/Getty Images) Nadella also said advanced AI development should not be controlled by a handful of companies. "The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia," he said. "This is the approach we are taking: broad access and choice at every layer of the AI stack; enterprise control of learning loops and models; and the ‘Code of Conduct’ that underlies our own first party MAI models that we’ll publish tomorrow for public consultation." His comments come as debate intensifies over the rapid pace of AI development and the risks posed by increasingly capable systems. MICROSOFT CEO HAS A WARNING ABOUT THE AI RACE Anthropic CEO Dario Amodei said that there are "real dangers" associated with AI development. (Anna Moneymaker/Getty Images) "I won't lie to you – there are real dangers," Anthropic CEO Dario Amodei said in an interview with CBS News . "And I think for too long the industry lied to people about the fact that this technology had risks." GET FOX BUSINESS ON THE GO BY CLICKING HERE Last week, former Anthropic researcher Jacob Coxon warned on social media that Anthropic and OpenAI are "gambling with our lives" by pursuing self-improving superintelligence, adding that AI has a greater than 10% chance of "kill[ing] all humans" within "the next decade." FOX Business' Robert McGreevy contributed to this report.