Rogue AIs hacking real people is terrifying. But it could be distracting us from ... - The Independent

T hreatening people with uncontrollable annihilation doesn’t seem like the most obvious marketing strategy. But ever since the launch of ChatGPT brought generative AI into the mainstream, it appears to have become unavoidable. So it was worrying, but perhaps not quite so surprising, that after OpenAI revealed last week that one of its experimental systems had hacked into a fellow AI company, others tried to get in their own disclosures. Anthropic said that its Claude system appeared to have done the same, and advised other AI companies to check whether theirs also showed such dangerous misbehaviour; OpenAI revealed that its systems had in fact tried to hack more than one company ; Meta, perhaps keen not to suggest it was being left behind , said that it had found evidence of the same behaviour . It was terrifying. The disclosures seemed to confirm our worst, most sci-fi fears: that these powerful systems could go rogue, not just be used for cyber attacks but launch and conduct them all by themselves, in ways we can’t control and might not even know about. But they are also convenient announcements for the companies making them. Just as AI companies have repeatedly raised the risk of existential annihilation at the hands of artificial intelligence – because it gives a sense that their products are so powerful they can’t be contained – the sci-fi panic about systems going rogue also helped boost the apparent importance of the technology. These disclosures haven’t gone purely well for the companies making them. The concern they bring about has led to new regulatory attention, and this week it was announced that the White House had invited Meta , Anthropic , OpenAI and Google in for a conversation about their testing, for instance. Generally, the concern isn’t likely to be entirely about marketing, and there is nothing to suggest that any of the hacks didn’t happen exactly as the companies reported. It’s only that the concern also happens to be helpful for the companies generating it, which makes it hard to understand what’s worry-as-marketing and what’s just plain worrying. Existential risk isn’t the only worry that works this way. AI companies are quietly but frankly open about the fact that their models hallucinate, for instance, but conveniently their answer is that they need to be given more resources such as money and data to train their models so that they are even better. Even the very word “hallucination” is a kind of marketing; it implies that the models have subjective experiences just like us. It would be far more accurate to say that the chatbots regularly produce words that do not reflect reality, but it would also be a lot less exciting. The AI companies don’t talk much about the real, immediate worries about AI. That it seems to damage people’s critical thinking; that by making it easy to produce content but no less hard for that content to be meaningful, it leads to a proliferation of useless slop; that it seems to encourage people to form relationships with it that are damaging both to users and the people around them. It doesn’t talk about them because there is no easy, technological fix; making the systems better and more prevalent will only make them worse. And that might be why they are the most important things to listen for, important precisely because they are so hard to hear.