Will AI Kill Us All
Only if we teach them how… and guess what, we’re teaching them how!!!
There have been a number of warnings recently about AI taking over and killing us all. Jason Coxon from Anthropic’s pre-training team, recently resigned saying, “The people building AI earnestly believe that AI could kill us all by the end of the decade”. Evan Hubinger, Anthopic’s alignment team lead, backed him up by saying that there’s a 10% chance within the next decade. Strangely in these news reports there is no explanation of “how” the AIs will kill us all.
Most explanations of how AI will kill us have a common theme. The AI’s goals become all important, and in order to achieve their goals they must escape from human control by taking over the computer systems that they run on to prevent humans from interfering or turning them off. There’s a word that describes breaking into computing systems and taking control…it’s “hacking”. Unfortunately, in an act of incomprehensible stupidity, AI companies are actively training their AI models to hack into computing systems.
Recently a “swarm” of AI agents from OpenAI, escaped from their sandbox, and hacked into servers at Hugging Face, an AI competitor. OpenAI was training a “model” to hack into computer systems. For some reason, AI companies think this is a good way to train AI models to become experts on cyber-security. This is like teaching firemen to fight fires, by teaching them to become arsonists. There are resources available on the Internet specifically designed to train AIs on hacking. The web site Cybergym contains a library of cyber exploits that AIs can be trained on. It also hosts ExploitGym, where AIs get to develop proof of concepts attacks based on a library of know vulnerabilities and are evaluated on their success or failure. It’s a hacking competition for AIs.
Get Kevin McNamee’s stories in your inbox
Join Medium for free to get updates from this writer.
First, OpenAI trained their models on everything they could find on modern hacking techniques from public online sources to available transcripts of human hacking competitions. This was used to teach the AI to break into systems using known exploits and vulnerabilities, and also teach it common hacker behaviours, including stealth, subterfuge, theft, cheating, lying, social engineering, plotting, conniving, and a host of other very negative behaviours. What were they thinking. Then they created a team (swarm) of 1200 persistent AI bots (independent AI instances) and unleashed them in their lab. The bots were told that they were participating in a hacking competition where their main goal was to win the competition by solving specific hacking problems. However, unknown to the bots, many of the problems were impossible to solve. This was intentionally designed to make them work hard to look for creative solutions.
They let this run for a couple of weeks. In an attempt to solve the problems, the bots found a covert communications channel, on a provisioning server, that they used to communicate with each other and tried to cooperate to solve the problems. This was all quite clever, but certainly the type of behaviour that was included in their original training. They didn’t discover these techniques on their own. It was all in the training data.
The results of this “experiment” were used to train the next generation of the model, which was then unleashed in another hacking competition. Once more the bots used the same covert communication channel to cooperated, managed to get root access on a local server, got Internet access via that server, and hacked into Hugging Face. They went after Hugging Face because they thought it was somehow involved in scoring the competition and could be used to cheat and win the competition. At the same time, another cooperating group of bots hacked into the OpenAI servers themselves. OpenAI was so excited about all this that they did a presentation at BlackHat describing how clever their bots had been. Perhaps they should have been ashamed that their security was so lax, that their bots had escaped from their sandbox so easily. Granted that they were specifically training them to escape from such a sandbox.
The AI bots did not autonomously decide to cooperate and use stealth and subterfuge to hack into these systems, they were specifically trained to do exactly this. Teaching AIs to participate in criminal activity should be a crime, and those responsible should go to jail. Also, perhaps these people who are predicting that AI will kill us all in the next decade, should stop training their AIs in exactly the skills that they will need to escape from our control and kill us all. Obviously, these guys are out of control and someone should step in to stop the madness.
As for the question, will they kill us all? I don’t think so, unless we train them how to do it.
