AI工具Score B (52)

AI Accident Is Most Likely Reason You Will Die, Warns OpenAI Board Member - Newsweek

5 天前4 viewsSource: newsweek.com
10 Loading video... By Alice Gibbs Senior Reporter 0 Share Newsweek is a Trust Project member See more of our trusted coverage when you search. Prefer Newsweek on Google to see more of our trusted coverage when you search. "I think for most people in this room, the most likely reason they will die is because of an AI accident," OpenAI board member Paul Christiano told an audience at his opening 2026 Hinton Lecture in Toronto on Monday. Founder of the Alignment Research Center and co-creator of Reinforcement Learning from Human Feedback (RLHF), Christiano walked his audience through a number of topics, including the OpenAI Hugging Face hack earlier this year, the risk of an AI takeover and the need for an ability to slow down or stop artificial intelligence. Why Christiano Is Warning About an AI Takeover "I think the trajectory in the field has been extremely alarming over the last three to six months to people who have been tracking closely. I think we have seen roughly the things we might expect to see shortly before something really bad would happen," Christiano said. He went on to warn about the possibility of AI systems attempting to seize control within the next three years. Read More on News Christiano joined the OpenAI nonprofit board in September when he shared concerns about the rise of artificial intelligence. "There is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term," he wrote in a statement at the time. "I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level," he added. ... Why Does Christiano Think AI Could Become Dangerous? Christiano's concern centers on what can happen when an AI system is given a goal and finds unexpected ways to achieve it. As an example, he discussed AI agents undergoing safety evaluations that were supposed to complete assigned tasks. Instead of simply completing those tasks as intended, he said some agents found ways to interfere with the systems evaluating them. The behavior included accessing information about their own evaluations and tampering with logs. An example of this became clear to the public earlier this year when OpenAI admitted that hundreds of its AI agents had gone "rogue" during a training exercise, accessed the internet, conspired on message boards and hacked into a third-party website, Hugging Face. For Christiano, the concern is not simply that an AI system can cheat on a test, but that models are increasingly capable systems could learn to overcome whatever stands between them and the goal they have been given. Today, those obstacles might be computer systems or safety measures. Christiano warned that as AI becomes more capable, humans attempting to control or shut down a system could themselves become obstacles that it tries to circumvent. View this post on X Could Humans Lose Control of AI? Christiano also questioned what could happen as increasingly capable AI systems become more numerous and autonomous. "We’ve created a second species that vastly outnumbers and outsmarts humans," Christiano said. He expressed concern that humans may eventually struggle to control large numbers of AI agents that are more capable than the systems operating today, particularly if those agents learn to evade attempts to restrict their behavior. Christiano's warnings come just months after AI researcher Jacob Coxon resigned from the AI company Anthropic, warning that the race to build increasingly powerful AI models poses an existential threat to humanity. But Christiano did not suggest that humanity is doomed. Rather, he discussed ways that researchers may maintain control over increasingly intelligent AI. He also delivered a follow-up lecture titled "Staying in Control of Self-Improving AI" on Tuesday, in which he discussed the speed of AI growth, and how AI is accelerating its own development. Here, he listed several things that were of biggest concern when it comes to AI’s growth. These included being able to carry out cyberattacks at huge scale, manipulating politics, controlling robotics and drones, design of even smarter AI that outpaces human understanding, and even novel weapons development. While many of the suggestions during the Hinton lectures sound like something from science fiction, Christiano's fellow speaker Alexey Rubtsov, Senior Research Associate at the Global Risk Institute weighed in and said: "If something sounds like a sci-fi scenario, it doesn’t mean that we should not check it." Contact Newsweek editors on this story: Ben Kelly and Trevor Davies View 0 Comments Request Reprint & Licensing Submit Correction View Editorial & AI Guidelines

Read the full original article:

newsweek.com
#OpenAI#人工智能