Moloch, Disaster Monkeys, and Gremlins
Humanity is on a ship, and no one is at the helm.
Think of humanity on a ship. It’s a large ship, with many working on it. Most are trying to push it faster, others move from one side to another, causing the ship to change direction. At the helm is a wheel, to steer the ship in a certain direction. There’s a great deal of fog around, but a few navigators have made it their practice to try to figure out where they are, and where they’re going. Unfortunately, they are too few, and are often ignored. To make matters worse, there seem to be a number of storm clouds ahead. And the helm? No one is at the helm. The ship has no intentional direction it is taking.
Some of the navigators have noted there is good reason to suspect there are icebergs ahead, and that they threaten to damage or even sink the whole ship. They try to warn various members of the ship, but unfortunately are overheard by someone: the Disaster Monkeys.
What are Disaster Monkeys?
The term was first coined by Elizer Yudkowsky. When you tell Disaster Monkeys something is dangerous, such as a poisoned banana, they want it. Because, to them, dangerous==powerful. Unfortunately, many times in history, this has been the case, so you can’t say Disaster Monkeys have always been wrong in their reasoning.
The disaster monkeys love icebergs, and anything dangerous, because they think such things could give them more power on the ship. They have not thought about how they could command a ship that is sinking, but the allure of the iceberg distracts them too much to worry about this. Mostly, they only worry about other Disaster Monkeys, and what horrors might happen if another monkey finds an iceberg first.
So now the navigators have to try to organize the ship and try to get people to take the iceberg problem seriously and prepare, to try to figure out where the iceberg is, how best to spot it, how far they are from it, the speed the ship is currently at regarding how fast they will hit it, and how to navigate the ship safely around it. And they also have to deal with the monkeys, who have now taken a sudden interest in steering the ship directly toward the iceberg. The closer you get to the iceberg, the more monkeys appear, all fighting with each other over the wheel. What begins as just a few, soon becomes a whole swarm. Welcome to your new job.
Humans are really bad at their job. Look out for each other, make things better for everyone, and try not to get everyone killed along the way. It’s not really that they’re bad at the tasks. Humans can do this sort of thing pretty well when it’s just a few other humans to look after. It’s just that they’re really bad at doing it at scale. Anything that needs a huge amount of intentional human coordination is going to be very, very difficult. This is made more difficult by a certain entity who’s been having some fun playing games with humans for thousands of years: Moloch. As ancient malevolent gods go, Moloch’s a simple guy. He likes getting people to play games with each other they otherwise wouldn’t want to be playing. Remember as a kid, you and a group of other kids all pushed each other into doing something dangerous, something you all probably didn’t want to do, but didn’t dare openly say because you didn’t want to appear chicken? Yeah, that was Moloch.
Moloch doesn’t just play with humans. He sometimes gets male Mooses (Meese?) to compete with each other for mates, by making their antlers grow out of control, until they get stuck in trees. Or gets Birds of Paradise to compete for mates by turning themselves all kinds of ridiculous colors, making them stand out to predators more, and lose all respect from others in the bird community who can no longer take them seriously.
But Moloch has been getting really wild with humans lately. He was always around, always causing trouble, but things have been getting a little out-of-hand lately. It started before World War II. Various scientists warned of the danger of fission reactors, and the potential of an atom bomb. Moloch didn’t waste much time summoning the Disaster Monkeys. The monkeys quickly heard of all the dangers atomic warfare could bring, and set their sights on getting to such a dangerous thing before other Monkeys did. The American Disaster Monkeys did it first, deciding if someone was going to destroy the world, it ought to be monkeys who did it for freedom and democracy. But before a single nuke was even dropped, Soviet monkeys turned to their comrades and connected the dots on what those Yankee monkeys were up to, and decided if someone was going to destroy the world, it ought to be monkeys who stood for the Worker’s Revolution, not those capitalist imperialist monkeys. Then nukes got dropped, and all the other monkeys soon wanted in. They got to work building thousands, coming up with quicker ways to end the world, and better strategies for ensuring the world would stay ended.
But some of the monkeys weren’t having a good time, in fact, none of them were. They all wanted the poisoned banana to scare away other monkeys, but with all the poisoned bananas around, they were starting to worry someone might actually swap them for the non-poisoned ones. They were starting to think this whole destroy-the-world business was a bad idea. But they worried that any other monkey hearing them say so would take it as an initiative to seize control of the entire banana supply. Any monkey caught speaking of reducing the poisoned banana, or forming a dialogue with other monkeys to get records on where the poisoned bananas were, and how many, might be seen as showing weakness. All monkeys had to act as though they were perfectly willing to poison all bananas, even if it meant poisoning themselves. How else could a monkey maintain any power?
This is how Moloch plays. Get the humans to enter a game without fully realizing the implications. Once the humans realize it’s a game they’d rather not be playing, it makes it difficult for any individual player to exit without harsh penalties.
And some poisoned bananas are easier to obtain than others. The “nuke” banana, though coveted by many a monkey, contains the “uranium” ingredient, and various other difficulties that make it hard for many monkeys to get a hold of. Superintelligence, though, will likely have much easier ingredients for monkeys to grab, such as GPUs. This means that with AIs, especially code that can be downloaded off github and run on normal computers, have become a breeding ground for disaster monkeys. Facebook AI’s leaked Llama network quickly had many monkeys tweaking it to make it more dangerous. The same case for OpenAI’s GPT4’s plugins, or Anthropic’s Claude. When there are a whole lot of disaster monkeys at play, some might simply do things just to see if they can, such as making additions to the base model to make it as dangerous as possible. Even Moloch can’t control so many monkeys when they decide to dance.
And then there are the Gremlins.
The problem with Gremlins is they’re very curious, clever even. Under normal circumstances, Gremlins can even be beneficial. They love to take new technology apart, and find some new way of using it beyond its intended purpose. Many times there might not even be much of a reason for it. It’s not as much about trying to determine if they should do something, just if they can. Unfortunately, as Gremlins get their hands on more and more powerful AI models, chaos will likely ensue. This isn’t even a Moloch game at this point. This is a Gremlin game. Gremlins don’t mind sharing everything they discover for the whole world to see, since it’s not about strategic advantage. It’s much more about fun. We’ve already seen some Gremlins’ idea of fun, such as taking apart LLMs, and creating autonomous LLMs with the task of killing all humans. At this stage, this is perfectly harmless. Some might even call it cute. But as AI transforms, like Gremlins, something that starts as cute might quickly turn ugly.
Some Gremlins have absolutely no thought of the consequences of their actions. It’s not about money, power, or ideology. Some Gremlins just want to watch the world burn.
Enter the Chaos Gremlin. They will take anything apart and find a way to dial the Chaos setting up to 11. They have circles of other Gremlins all working toward making things as malicious as possible. I’ve had experience in Cybersecurity, I’ve seen these Gremlins at work. Once a technology is on the internet, it falls into the hands of these Gremlins. If there was a button that could destroy the world available online, one of them would have pushed it. Any company releasing technology to the public needs to remember that their technology is now in the hands of the Chaos Gremlin. Any technology that risks being leaked to the public risks being leaked into the hands of these Gremlins. The safest thing to do when creating a dangerous technology is not to build it in the first place.
