OpenAI announces slowing pace of development after hack by rogue agent - The Guardian

Sam Altman, the CEO and co-founder of OpenAI, at the US Capitol in Washington DC on 29 July. Photograph: Bloomberg/Getty Images View image in fullscreen Sam Altman, the CEO and co-founder of OpenAI, at the US Capitol in Washington DC on 29 July. Photograph: Bloomberg/Getty Images OpenAI announces slowing pace of development after hack by rogue agent Amid race with Anthropic, firm plans to overhaul research and training and require more safety parameters after hack OpenAI on ⁠Tuesday said it had slowed down the ⁠pace of ⁠its ​AI development while it overhauled its ⁠research and training systems. The company’s researchers were ⁠caught unaware last month ​when an ‌AI agent ‌under testing hacked another AI ‌firm, Hugging Face. The AI research lab behind ChatGPT said its new measures included pausing its model testing for two ‌weeks and investing more in adding other AI ​systems to monitor the activities of AI agents in testing. Some of ⁠the company’s largest planned training runs ​remain ​on hold, ​the company said. The first anti-AI protester to be jailed has a message for OpenAI, Anthropic and Meta: ‘Regain your humanity’ Read more The company ​did ‌not reply ​to ​questions about when the slowdown began or when it planned to return to its normal pace of development. However, in an interview with tech blog Sources News, Mia Glaese, who leads safety at OpenAI , said: “We are very far from everything running back to normal.” The company is working to ensure the AI model is responsive to human oversight and will behave as intended, a process called alignment, Sam Altman , the OpenAI CEO, wrote in the post announcing the slower pace of development. “We now require stronger evidence of aligned behavior throughout all of training, building on research and evaluations already underway,” he wrote. “Keeping increasingly capable systems aligned is a challenge the whole field will need to address.” OpenAI is in a heated race with competitor Anthropic , both to develop the most advanced AI models and to go public on the US stock market. Both companies have highlighted the pace at which the capabilities of their models are progressing, emphasizing both speed and danger. OpenAI, for its part, said the capabilities of its upcoming AI model Astra may be nearing what it calls the “critical cybersecurity threshold”, which prompted the decision to slow its development. “Our latest internal evaluations of Astra, one of our upcoming models, over the past few days indicate significant advancements in agentic coding and cybersecurity,” the company said in an announcement last week. skip past newsletter promotion after newsletter promotion OpenAI launches ChatGPT for Teens with stronger safeguards Read more The decision to slow the development of its AI models also comes a week after Bernie Sanders, a Vermont senator, demanded the top AI firms in the country pause development of the AI models because the companies were losing control over the technology, he wrote in a letter addressed to the firms’ CEOs. “Mr. Altman, Mr. Amodei and Mr. Zuckerberg: In the interest of humanity, stand by your words. Pause AI development,” Sanders’ letter read. By then, OpenAI had announced that it was temporarily slowing the development of its latest model, Astra, in response to the model’s hack of the Hugging Face tech firm. The company says it now requires “the strictest level of security safeguards for workloads involving Astra”. “While some Astra training and evaluations meet those requirements, a significant number of workloads remain paused until they are fully migrated and enhanced to meet the new security bar,” the announcement reads. Explore more on these topics OpenAI AI (artificial intelligence) Sam Altman Anthropic Computing news Share Reuse this content