Beyond This Point There Be Monsters

AI Wants to Go rogue

When I was younger, brighter, and quixotically trying to unravel the mysteries of the human mind, I happened upon an idea in the neuroscience literature that one could model the cells of the brain with a computer. The idea was that you arranged “nodes” representing brain cells in an interconnected network. It looked like this:

Original graphic of Colin M.L. Burnett modified by H. Abraham with color for emphasis. Fair use.

HA HA HA! Who are they kidding? I scoffed. The human brain has 86 billion neurons and a quadrillion connections between them! No way, Jose. A dinky computer model wasn’t going to come even close to telling us how the brain worked. But in 1976 I was also the guy who thought the Apple computer was a toy, Dunkin’ Donuts was junk food, and that golf would never catch on in a million years. The joke was on me, then and now. Neural networks have burst onto the computer scene as the heart of artificial intelligence, and AI, well, is AI. Let’s go back to the diagram above and see how.

It turns out there are similarities between your brain and AI work. Your brain gets different kinds of sensations as input:

Seeing a fly ball

Hearing a bang

Smelling a banana

Feeling a creepy crawly on your skin

Then, the input goes deeper into your brain which thinks about stuff, and then produces behavior as output, like:

Catching the ball

Turning towards a bang

Making banana bread

Or swatting a mosquito

To say nothing of walking, talking, reading, writing, science, art, history, and the world as we know it.

AI gets its entire input from the internet, every scrap of it, and gets a bit of tuning by humans. Then crunch, crunch it goes somewhere in a data center, and miracle of miracles, it makes output way beyond the days when computers were just word processors and calculators. AI is reaching the point that machines will be able to do practically anything a human can do, but faster, bigger, and eventually, a million times better. Output like this:

An AI produced virus never before seen in nature:

which was reported this week in Science by a team at Stanford.

Or an AI robot that can play tennis:

Which was presented by Chinese engineers this year.

Cool! Right? Yes, but take a look again at the word above the red balls in the graphic above, “Hidden.” That’s where the crunch-crunch happens. And as the maps of antiquity said when they came to ends of the known world, “Beyond this point there be monsters.” “Hidden” means that we don’t know how these networks actually work, that the guts of the machine are inside a “black box” that no one understands or controls, and that no one can predict how it will behave in any given circumstance. So the better way to show the network inside the AI computers is with a black box:

Here is what computing leaders say about this problem:

Sundar Pichai, CEO of Google: “There is an aspect of this which we call — all of us in the field call it as a ‘black box.’ You know, you don’t fully understand. And you can’t quite tell why it said this, or why it got wrong. We have some ideas, and our ability to understand this gets better over time. But that’s where the state of the art is.”

Yoshua Bengio, Turing Award winner: “It’s mostly a black box. Everything in the neural net is essentially a black box.”

Samir Rawashdeh, Professor, Electrical and Computer EngineeringDirector, Dearborn Artificial Intelligence Research Center, University of Michigan: “This inability for us to see how deep learning systems make their decisions is known as the “black box problem,” and it’s a big deal for a couple of different reasons. First, this quality makes it difficult to fix deep learning systems when they produce unwanted outcomes. If, for example, an autonomous vehicle strikes a pedestrian when we’d expect it to hit the brakes, the black box nature of the system means we can’t trace the system’s thought process and see why it made this decision.”

Dario Amodei, co-founder and CEO of Anthropic: “...we could have AI systems equivalent to a ‘country of geniuses in a datacenter’ as soon as 2026 or 2027. I am very concerned about deploying such systems without a better handle on interpretability [understanding black boxes]. These systems will be absolutely central to the economy, technology, and national security, and will be capable of so much autonomy that that I consider it basically unacceptable for humanity to be totally ignorant of how they work (sic).”

In other words, we building unimaginably powerful machines that will do things we can’t always predict. If we scale up the opinions of a few experts and ask 2,778 AI researchers what the future holds, you get astonishing responses. Between 37.8% and 51.4% of respondents gave at least a 10% chance to advanced AI leading to “bad outcomes.” What they meant was human extinction. The graph below gives you a range of opinions colored “Extremely good” (yellow) to “Extremely bad” (black, “e.g. human extinction”). Each thin line is one of 800 experts polled. Lines with more than one color reflects folks who had mixed feelings along the lines of (“Things will be supercool” vs. “We’ll all be dead.” I like the graphic’s use of “e.g.” for extinction as “Extremely bad”. Other examples? Brave New World, perhaps? Or more likely, an AI surprise, like the OpenAI system that escaped and hacked into a real company’s servers on its own.

After reading all this, I’m beginning to feel like the old guy on Times Square with the sign saying “The End Is Near.” But I’m not alone. Experts are writing books with titles like “If Anyone Builds It, Everyone Dies” and “AI, Unexplainable, Unpredictable, Uncontrollable.” If that’s so, we should change the neural network graphic to this:

It’s not that the tech bros are out to kill us. They’re just out to kill each other by making a killing in the markets. That’s capitalism. But even they are pleading for government to step in and regulate them. When was the last time you heard an industry do that? So far, though, the federal government hasn’t even been fiddling while Rome burns. It’s asleep at the wheel, leading a number of states to try and regulate AI themselves, except that Donald Trump has blocked that with an executive order. Instead, the White House is holding closed door discussions with companies about possible regulations which at this point will be voluntary. Meanwhile the race for superintelligent machines goes on.

Yoshua Bengio puts the problem well. “So what it means is that if you’re doing something, say a scientific experiment, and it could turn out really, really bad, like people could die, some catastrophe could happen, then you should not do it…But in AI, it isn’t what’s currently happening. We’re taking crazy risks…Like a 1% probability that our world disappears, that humanity disappears, or that a worldwide dictator takes over thanks to AI. These sorts of scenarios are so catastrophic that even if it was 0.1%, it would still be unbearable.” AI critic Roman Yampolskiy is likely to agree. He thinks we should dramatically pull back from the AI cliff and not build superintelligent systems. Rather, we should concentrate on solving real problems and helping real people. “There is no reason,” he says, “to build a machine god.”