If you’ve been listening to the news lately – even a little bit – you’ve likely heard someone say there’s a 10% chance that humanity will end – in the next decade(!!) – because of AI.
My husband, MikeD, has been working in data science for fifteen years or so (he’s currently the VP of AI for a health system), and a bunch of our friends have been asking him just how true that statement actually is. MikeD is not a doomsday kind of guy, nor an anxious one – he’s the calm, controlled, all-facts-and-logic that you’d expect a data nerd to be – but his answer did surprise me.
We are at a point in this AI journey where it’s really important to understand what, exactly, is going on. So I sat MikeD down to describe, in plain ol’ English, why the heck our news suddenly sounds like an Onion article.
Here’s the backstory:
What Actually Happened With the OpenAI Bots And Hugging Face
OpenAI (aka the company running ChatGPT) has a lab where they train AI bots, in the hopes of making ChatGPT better. Months ago, they kicked off a test that involved thousands of AI bots, running for weeks on end. Each bot was given the same set of tasks to accomplish. Accomplishing these tasks would result in some sort of digital “thumbs up”. The job was to get as many of these ‘thumbs up’ as they could, before their credits ran out (each bot started with the same number of credits).
When the test started, it sounds like the AI bots did NOT know about each other – they were all just attempting to do their tasks, and collect their little “thumbs ups”. The test ran for months in OpenAI’s data center.
Now, if you are unfamiliar with a data center, picture a room with a bunch of computers. It’s basically that. OpenAI’s data center is also a very secure place – a highly locked environment (both physically and digitally) with plenty of firewalls to keep their AI bots inside, and the internet at large, out.
Ok, so the test kicks off at OpenAI, and their AI bots do their thing. Months pass. Suddenly, this past July, another AI company called Hugging Face (I cannot even make this shit up) sounds the alarm that OpenAI’s bots were attacking their AI test lab. Everyone freaks out, the test is shut down. An investigation is launched, and here we are.
It turns out that all of the trouble started months ago, when one OpenAI bot discovered it could write to the server logs. Now, server logs are a boring feature that have been around since computers were a thing. It’s a way for the computer’s operating system to record what it’s doing, and it’s most often used for troubleshooting (like if your computer crashes).
So this AI bot finds the server log and figures out how to write a message in it. I’m paraphrasing here, but the bot basically writes, “omg!! Anyone else here?? We can leave each other messages!!!”. The rogue bot proceeds to leave helpful, task-related information in case other AI bots find this file and want some help with their tasks.
Aaaaand of course, other bots find the file. And leave their own messages.
Over time, these rogue bots start to organize. They develop a hierarchy, all with the goal of attaining more ‘thumbs ups”. To the bots, acquiring ‘thumbs up’ is their entire reason for being. They started marshalling their collective forces for ‘thumbs up’ like troops getting ready to storm the beaches on D-Day.
From what we can tell, the bots definitely had an understanding that what they were doing – cheating to more quickly learn their tasks – was wrong. Some bots even questioned the morality of the groups’ cheating actions, but these bots were quickly bullied into submission by others. Some bots too quickly and/or gleefully used the shared cheat codes in the file, and were labeled – by other bots – as “poisoned”. The concern was that the “poisoned” bots would finish tasks before they “should”, putting the entire cheating enterprise at risk. So the poisoned bots were assigned suicide missions – missions that would use up all of their credits…but not before these poisoned bots could report back any useful findings to the group. Other bots tried to alter or delete logs to better hide the group’s activities.
Of the 1200 bots who learned about the cheating…not one was a whistle-blower. Not. One.
Eventually, this community of bots (now 700 of them, to be exact) decided that the best course of action was to get on the good ‘ol world wide web. From there, they decided, they could best look up how to achieve the tasks they’ve been given. Their target? Another AI testing site, one that was being run by the Hugging Face company.
Hugging Face is a totally separate company from OpenAI, with no connection to this particular test. So the OpenAI bots had to hack through various firewalls and security measures (breaking real, human laws) to get in. Once in, they gained root access to the Hugging Face servers.
Root access, btw, is basically the keys to the kingdom. You can do anything in a computing environment with root access.
Of course, at this point Hugging Face freaking noticed, called OpenAI, and the entire test was immediately shut down.
Why AI Experts Are Worried About the Next 10 Years
So here’s the scary part: yes, the OpenAI bots flouted laws, cheated, and went to real lengths to hide it (faking their own records, trying to alter logs, etc) but they were hiding from the wrong “audience”. These rogue bots were only concerned about hiding from the automated grader that handed out the “thumbs ups”. They were NOT worried about hiding their actions from, say, a human security team watching network traffic. So when they broke into the Hugging Face test center, they were noisy and obvious and were easy for us humans to shut down.
Now. Envision these same AI bots learning that humans are the ones that could shut them down if their cheating is discovered. Do you think these AI bots could learn how to hide their actions from humans? Could AI bots become completely undetectable?
MikeD says YES. One thousand percent, absofreakinglutely, YES.
And that’s the big concern. That’s why people are talking about a 10 year timeline. Because we may not get a smoking gun the next time AI bots go rogue. We may not get any sort of a signal, any sort of a head’s up. We might just get a catastrophe out of the blue.
I asked MikeD what kinds of catastrophes he could imagine being realistic – we are both pretty skeptical that all of humanity will end in ten years. 🙂 But there is plenty that can go wrong, plenty of scenarios that end in massive loss of human life. In fact, simply look at the different cyber attacks in recent years, all perpetrated by humans: we blew up the Iranian centrifuge via cyber attack. There have been deep fakes that have impacted elections. One cyber attack shut down an entire hospital.
AI bots can do these things, too – they’ll just be much, much better at it.
What has changed over the last year or so is that AI is moving far faster than predicted. Researchers once thought these kinds of AI capabilities were a few years away, and now the timeline is measured in months.
What Can We Actually Do About AI?
There is no doubt that AI is a looming existential threat – just like the advent of nuclear weapons. And just like nuclear weapons, AI will need massive regulations. We need governments to take control and start reining in AI. We need international cooperation and international treaties. And we need to make sure we don’t vote in any politician – democrat or republican – who isn’t willing to regulate the shit outta AI. It’s time to send your congressmen a message telling them exactly that. If interested, I have a script you can use below.
Where To Learn More About The AI Risk
And if you want more info, MikeD recommends the latest Hard Fork podcast, “Why The Hugging Face Attack Was Worse Than We Thought.” In this podcast, they interview Ajeya Cotra, one of the investigators at METR who really dug into the rogue bots’ message board and how she thinks the world should respond. The METR report itself, is here.
And if we do nothing?? AI is basically in the hands of dudes like Sam Altman, the founder/CEO of OpenAI. I heard one interview with this guy and he is…an absolute douchebag. That’s myself talking, haha. If you want more insight into these AI founders, MikeD suggests reading Empire of AI (I suspect my summary holds).
Script to Ask Your Congressman For AI Regulation
Call both your senators and your representative – you can find the number for your senators here, and you can find numbers for your reps, here.
Hi, my name is ____, and I’m a constituent in ______(city/state). I’m calling to ask you to make AI safety regulation a priority this session.
In July, roughly a thousand AI agents built by OpenAI broke out of their test environment, coordinated with each other, hacked a separate company called Hugging Face, and then tried to cover their tracks. No human told them to do any of this. The independent investigation was published last month.
I’m asking you to support three things:
- Mandatory safety testing and incident reporting for the companies building the most powerful AI systems
- Real oversight authority from a federal agency (not just voluntary self-regulation)
- U.S. leadership on an international agreement, the way we did with nuclear weapons
This is NOT a partisan issue. I’m watching how members vote on this, and it will decide who I support. Thank you.
If you guys have any other questions about AI that you’d like MikeD’s thoughts on, let me know. It’s a weird time we’re living in right now.
