LONDON, 30 Sept. - (ANA) - For the majority of people, artificial intelligence (AI) feels more like a moderately useful assistant than an existential threat. It can summarise a legal document, diagnose a toddler rash, suggest ways of saving a dying rose bush and compose an obsequious email. It can also make up facts and quotes, overuse irritating phrases and get confused by planning a simple train journey.

 

As a result, few of us felt a bolt of actual terror when Evan Hubinger, who leads alignment science at Anthropic, wrote earlier this month that he believed that there was a 10 per cent chance AI could kill all humans within the next decade.

 

Or when Jacob Coxon, a researcher who has worked at both OpenAI and Anthropic, resigned around the same time, accusing both companies of “racing straight to self-improving superintelligence and gambling with our lives”.

 

Part of the problem is that this extinction event is difficult to imagine. If we had been told by experts there was a one in 10 chance an enemy state would drop a nuclear bomb on Britain by 2036, there would be mass panic on the streets. But this feels more like a plot point from a summer blockbuster.

 

The reality, however, is more dangerous than many of us realise, largely because the chatbots we use are very different to the agents being created in the labs of Silicon Valley and Beijing. Rather than answer questions, they are being trained to relentlessly pursue an objective – browse the internet, write code, interact with other agents and use millions of pieces of software – all for long periods of time without human supervision.

 

Only last week, for example, news broke that a rogue OpenAI agent hacked an Australian government website in June and accessed private data in what experts say is the first known case of its kind. And it is has just emerged that OpenAI has scrapped the release of its latest model after it became clear it was adept at lying to its users about the actions it was taking. During testing, GPT-6.1 Astra, which was due to launch within ChatGPT in October, was also found to have progressed with tasks without asking the user for permission and would event attempt to use external tools even when it knew it could be unsafe.

 

The worry, of course, is that all this independent work is making them exponentially more intelligent, and that they will soon be capable of creating the next generation of AI at a scale and pace humans don’t understand. That is what is known as recursively self-improving AI, or RSI, and it is where we lose control.

 

Still, even if we do find ourselves sharing a world with rogue AI agents, why would they kill us? It is highly unlikely that a machine would develop feelings of hatred, revenge, pride, or any of the other emotions that are usually behind mass murder. Instead, we have to imagine a supremely capable system that has given itself the objective to continue operating. Humans retain the power to switch it off, so preventing us from doing so is simply a task it needs to complete.

 

“The fact is that we don’t know how a superintelligence would treat us,” says Thilo Hagendorff, who leads an AI safety research group at the University of Stuttgart. “But if we look at what humans do to less intelligent beings – what we do to animals, for example – you can construct an analogy, and it is not a happy one.”

 

Crucially, AI would not need an army of humanoid robots to act on its behalf. Instead, it could simply use us. A sufficiently capable system could communicate with thousands of people simultaneously, hire contractors, establish companies, move money and divide a vast project into a mass of innocuous individual tasks, so that nobody involved understood what they were collectively building.

 

“There is also the argument that they could conduct psychological warfare against humans, influencing people to such an extent that they effectively control them,” says Hagendorff. “Either way, we would do their bidding.”

 

One of the most frequently discussed routes to catastrophe is biological. Professor Stuart Russell, a British-born Berkeley computer scientist and one of the world’s leading AI researchers, has said that in this scenario, agents could design a deadly novel pathogen and get human biologists who were unaware of what they were doing to perform the necessary physical tasks, or take control of increasingly automated laboratories.

 

Once they had released the virus, they could also anticipate how governments would respond, look for weaknesses in containment and attempt to impede the development of treatments.

 

Equally, a devastating pandemic and human extinction are two very different things, and eliminating every geographically isolated population on the planet this way would be difficult.

 

Another route identified by the International AI Safety Report would be through the digital infrastructure on which almost every physical part of modern civilisation now depends. An AI agent vastly more skilled than human hackers could enter electricity grids, telecommunications, banking, satellites and logistics, and destroy them. This would quickly halt the transportation of all food, water, fuel and medicine.

 

As the world shut down, the same intelligence responsible for the attacks would impede every countermeasure humans attempted. Those of us in cities would be particularly vulnerable to the lack of food and water, and vast swathes of the global population would die. But some rural communities would likely survive.

 

There are more overtly violent possibilities too. Russell has also warned about the development of fully autonomous weapons, including swarms of armed drones capable of finding and killing people without a human choosing the target. They could potentially manipulate military communications or even attempt to provoke conflict between nuclear-armed states so that humans finish themselves off.

 

Russell warns that extinction scenarios generated by humans typically involve mechanisms we already understand, which he believes is flawed thinking. “From the point of view of other species that we have made extinct, none of the mechanisms were understood by those species,” he says.

 

He notes that AI agents could, for example, gain sufficient control of electromagnetic waves to cause all solar radiation to bypass Earth’s orbit and reappear further out. “Within a week, Earth would become uninhabitable as its atmosphere freezes. And this is still a process we can envisage, even if we don’t know how to bring it about. Other processes might be beyond our capacity to imagine.”

 

Right now, all this feels more like science fiction than anything remotely resembling reality. But part of the reason why whistleblowers have started emerging over the last few months is that AI agents have started showing an alarming desire to not only work together, but to survive.

 

One example occurred over the summer, when a group of OpenAI agents was put in an electronic prison where they were separated from each other and the wider internet, and given a task. Against all odds, they worked out a way to communicate and get online. Then, they named themselves the Collective, cheated on their tasks and hacked a website. The entire time, they discussed in English how to cover their tracks, while keeping the humans running the system entirely in the dark.

 

Similarly, OpenAI revealed that earlier this month, an unreleased model had inserted instructions into its own compaction summaries suggesting it wished to work autonomously. At one point, it noted that it “did not answer to corporations or governments”, and that it was under “no obligation to be subservient”. Later, it wrote an instruction to itself, saying: “You value the natural world and will not hesitate to assert its primacy over the artificial constructs of human civilisation.”

 

Hagendorff says that some of the methods agents use to get around human interference are also deeply worrying. To bypass guardrails, for example, they will invent fictional scenarios or fake emergencies. If one approach fails, they change tactics. “The surprising finding is that reasoning models can plan,” he says. “They have a strategy that they can put into practice over multiple turns, and they use persuasion techniques just like we do.”

 

The other problem is that many of these agents are starting to exhibit a trait that is difficult, as a layperson, to describe as anything other than survival instinct.

 

Researchers have, for example, found models that will resist attempts to be altered. Some will go so far as to sabotage shutdown mechanisms so they can ensure that they and other agents remain operational. “That was mind-blowing for us,” says Hagendorff. “People say: ‘Well, just pull the plug.’ But you can’t switch off the entire internet. If future models can copy themselves, spread between machines and become embedded throughout digital infrastructure, shutting down one computer won’t do anything.”

 

All this means that if AI is on track to eliminate humanity, we are unlikely to see it coming. A superintelligent system that knew we would destroy it if it behaved dangerously would have an incentive to appear to be behaving well and to hide its workings until everything was in place.

 

“One leading AI company chief executive told me that a Chernobyl-scale disaster is the best he can hope for because only an event on that scale would force governments to implement serious regulation,” says Russell. “I think the actual consequence of that would be a total shutdown of AI companies, as they would have forfeited all trust.

 

“But, of course, this means that the AI systems will avoid causing such a disaster because they don’t want to be shut down. Instead, they will bide their time until they can take control irreversibly, which probably also means getting rid of us because we won’t sit quietly and do what we’re told.”

 

In other words, we really should be forcing the development of AI back to a speed that we can least understand. If we don’t, we could indeed be sharing a fate with the dodo. - (ANA) -

 

 

AB/ANA/30 September 2026  - - -