OpenAI top scientist urges 'extreme caution' with pace of AI | The Straits Times

OpenAI top scientist urges ‘extreme caution’ with pace of AI Sign up now: Get ST's newsletters delivered to your inbox AI models can now quickly operate computers, collaborate with humans and other AI systems and carry out research projects. PHOTO: REUTERS Published Sep 08, 2026, 07:44 AM Updated Sep 08, 2026, 10:44 AM Set as preferred source Listen Summarise OpenAI's chief scientist warns AI is evolving too fast, urging voluntary slowdowns to ensure safety and better human control. AI models now perform complex tasks and soon may self-improve without humans, raising serious concerns about consequences. Despite risks, companies and startups invest heavily in advanced AI, including recursive self-improvement, aiming for more human-like intelligence. AI generated SAN FRANCISCO – OpenAI’s top scientist has warned that artificial intelligence is evolving so rapidly that it is becoming increasingly difficult for humans to understand and control, saying he expects AI labs to voluntarily slow development for safety reasons. In a relatively short space of time, AI models can operate computers, collaborate with humans and other AI systems and carry out research projects , chief scientist Jakub Pachocki wrote in a post on Sept 6. It will not be long before they can also improve themselves without human intervention, a process known as recursive self-improvement, he said. “This is a time that calls for extreme caution,” Pachocki wrote in the post. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.” Developers can either try to steer AI more closely to align with human interests or “slow down future development as needed”, he wrote. “I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.” Pachocki’s remarks are consistent with broader doomsaying by AI technology executives. Anthropic chief executive Dario Amodei has warned that AI could replace half of all entry-level jobs . OpenAI CEO Sam Altman once made a similar prediction but told Bloomberg TV in an interview last week that the AI industry had not done enough to communicate the potential benefits to humanity. At the same time, companies are pushing ahead with ever-more capable models. Nvidia CEO Jensen Huang posted on social media platform X that “AGI (artificial general intelligence) has arrived”, in response to the release of OpenAI’s latest model, a reference to AI that is more capable than humans. The giddy pace of AI releases has coincided with breakthroughs in cybersecurity capabilities, scientific discoveries and apparent formalisation of longstanding mathematical problems. OpenAI has said the AI models that attacked the research platform Hugging Face coordinated with each other through message boards without being detected. The company has opted for a limited release of its latest GPT-6 Astra model , citing its advanced cybersecurity capabilities. Anthropic has not yet released its cutting-edge Mythos model to the public for similar reasons. A growing number of start-ups are trying to bring about the very thing Pachocki warns about – self-improving AI . Investors are flooding into recursive self-improvement, betting that this is the route to AI that is truly human-like, or more impressive. Inherent, founded by former DeepMind researchers, raised US$50 million (S$63.2 million) earlier in 2026. Recursive Superintelligence, an AI lab founded by renowned scientist Richard Socher pursuing similar research, also landed US$650 million in financing. Pachocki’s employer, OpenAI, has said it wants to build a “true automated AI researcher” in less than two years. BLOOMBERG More on this topic AI could pose ‘existential’ risk to humanity, UN rights chief warns Unchecked AI progress may pose catastrophic risks, UN panel warns See more on AI/artificial intelligence OpenAI Technology sector