Skip to main content

Dumb AI Will Kill Us Before Superintelligent AI Has A Chance

Humans have been fascinated with the idea of a robot revolution since we first conceived of robots. Long before Skynet and the Terminator were imagined in the early 80's (?!?), Asimov wrote of the Three Laws of Robotics, warning us of the potential for intelligent robots to do us harm. Recently, with the rapid expanse of deep learning neural nets in the last few years, we hear more and more about how an artificial "superintelligence" (i.e., beyond human intelligence level) could wipe us out with as much difficulty as a human stepping on an ant. While I do think that superintelligence could potentially be an apocalyptic threat to humanity if it were ever to be created without proper safety built in, I am actually way more concerned about what I like to call "dumb AI". Which honestly can kill us right now if we're not careful.

What's "dumb AI"? Well, artificial intelligence, or AI, is a buzzword that spans many different technologies. The media tends to call any technology that does anything sufficiently complicated "artificial intelligence". In computer science circles, it means something different, but for our purposes, let's go with the popular view of "technology that does something complicated that probably used to be done by a human" (I am aware that this is incredibly broad). General human level AI would be a system that is as intelligent as an average human being. Superintelligent AI would be a system that is more intelligent than a human, perhaps by orders of magnitude. So "dumb AI" is what we have now - systems that are good at isolated tasks, potentially even better than humans, but that have no general intelligence.

Autonomous cars are an example of "dumb AI" - they're good at the single task of driving, which they handle by combining data from different sensors such as lidar and cameras, building a model of their environment, and using that model to follow the road and avoid obstacles. A difficult task, for sure, but your Model S is never going to become "self-aware", because the only thing it knows is how to drive. It has no opinions on current events, it can't make you a coffee (but one day it will probably drive itself to Starbucks to pick up a coffee for you), and it wouldn't hold up very well in a Turing test. It seems really "smart" that it can drive, but it is designed specifically for this one task, and has no general intelligence.

Teslas are amazing machines, but let's return for a moment to the fictional world of Terminator. In the first Terminator, a super strong, intelligent cyborg takes a hop in a time machine back to 1984 and starts kicking ass and taking names. This scenario is unlikely in the near future, obviously because of the time machine, but also because we're pretty far away in both robotics* and artificial intelligence. It would be cool if text-to-speech systems could do an Ahnold accent, though.

Terminator 2 was even more unbelievable - liquid metal time traveling robots, somehow with enough distributed intelligence in the molecules of metal that you could blow the thing to pieces and it would magically come back together. We're nowhere close with nanotech, sorry.

Which brings us to Terminator 3. A quaint little movie from 2003. Check out the start of the war:

An engineer reports Skynet "processing at 60 teraflops a second". Well, that was actually surpassed by a single supercomputer in the 2004 timeframe (, and distributed systems have way more combined processing power, although processing power by itself isn't enough to incept intelligence. More important than the depiction of Skynet's self-awareness, check out a scene from a few minutes later showing a killer drone called a "hunter killer":

In 2003, this was pretty fantastic to see. A drone with heavy firepower and automatic targeting! Holy shit!

Well, guess what. All the technology needed to build that exists right now. It might have been Hollywood effects then, but we have it now. Quad copter drones with precise maneuvering capabilities? Check. Automatic targeting systems? Check. Slap some guns on a beefed up quad copter and give it a relatively dumb computer vision system, and you basically have the hunter killer from Terminator 3. The main difference is that there's no superintelligent Skynet behind it.

But actually, that's precisely my point. The technologies needed to accidentally start global world-ending conflicts are here, not tomorrow, not today, but yesterday. Check out this "automated turret" system from 2010, which South Korea wanted to put on the DMZ (not sure if they ended up installing it). Here's the breathless videogame-like description from the article:

"The Super aEgis 2 is an automated gun tower that can find and lock on to a human-sized target in pitch darkness at a distance of up to 1.36 miles (2.2 kilometers). It uses a 35x zoom CCD camera with 'enhancement feature' for bad weather, in conjunction with a dual FOV, autofocus Infra-Red sensor, to pick out targets. 
Then it brings the pain, either with a standard 12.7mm caliber machine-gun, a 40mm automatic grenade launcher upgrade, or whatever other weapons system you want to bolt on to it, including surface-to-air missiles. A laser range finder helps to calibrate aim, and a gyroscopic stabilizer unit helps correct both the video system's aim and the direction of the guns after recoil pushes them off-target."

The BBC covered this company again last summer: Supposedly, the autonomous firing mode has been turned off, but that doesn't make this tech any less terrifying. First, it can simply be turned back on if requested by the client (apparently, mostly friendly Middle Eastern countries...). Second, the things have a friggin' network connection, and Korean companies are not very well-known for their internet security (sorry, every Korean bank). Can you imagine an automated turret with a .50 caliber machine gun being hacked?? Additionally, the tech for this is honestly not very advanced. It's an infrared motion sensor combined with a turret. It has no way to distinguish friend from foe - it just shoots at humans. You know, like in Terminator. The "senior research engineer" doesn't inspire much confidence, either, stating,
“Within a decade I think we will be able to computationally identify the type of enemy based on their uniform.”
First of all, talk about lowering expectations! I could build a very accurate classifier today using open-source technology and my four year old laptop that would be able to accurately distinguish between South and North Korean soldiers based on uniform. Which is fine and dandy, but if slipping past an automated turret is as simple as stealing some South Korean army uniforms, then you can see why this is "dumb AI". What if this thing accidentally fires a bunch of RPGs over the DMZ due to a false positive in its vision system and starts WWIII?

Actually, buggy vision systems have nearly killed us already, many decades ago. Here's just one declassified incident from the Cold War that we know about: The tl;dr version is that the Soviet nuclear missile early warning system erroneously detected a launch of ICBMs from the US, and Petrov decided that it was probably a mistake, and thus arguably prevented a "retaliatory" nuclear volley from the Soviets which would have turned the early 80's into a Mad Max apocalypse. The problem? The Soviet early warning system got confused by the clouds and some Soviet satellites and pegged them as incoming American ICBMs. Whoops.

So, while most people are worrying about their IoT coffee makers becoming self-aware and inciting grey goo scenarios, I'm worried about the tech we already have. Drones are getting better and better, cheaper and cheaper. Right now a motivated individual could build a Terminator 3 style hunter-killer with a modified quad copter, some light guns, and off-the-shelf vision systems. On the other end of the spectrum, "real" drones such as the Predator can run autonomously without too much difficulty, and those pack serious firepower. Auto-targeting turrets on the DMZ and other high tension areas. Increasing autonomy built into other parts of the military and supply chain. Autonomous systems acting on the basis of requests from autonomous satellites. Intelligence based on imperfect vision systems. We're putting way too much power in the hands of systems that are extremely precarious. And the consequences of a mistake could be devastating.

Artificial intelligence is going to change the world in fundamental ways. We can't even imagine how transformative some of these changes are going to be, and the next few decades are going to bring scientific and technological advances that were considered science fiction not long ago. I just hope dumb AI doesn't accidentally kill us before we have a chance to see it.


* Of course while I was writing this, Boston Dynamics released a new video of their humanoid robot.

Slap some guns on it, along with today's best facial recognition system, and you basically have the T1. Give them a network connection and some goals (i.e., kill humans), and maybe we're not so far away from the Terminator vision after all.


Popular posts from this blog

영어가 모국어인 사람들은 왜 한국어를 배우기가 어려운 이유

이 포스트는 내 처음 한국어로 블로그 포스트인데, 한국어에 대하니까 잘 어울린다. =) 자, 시작합시다! 왜 외국사람에게 한국어를 배우기가 어렵다? 난 한국어를 배우고 있는 사람이라서 이 문제에 대해 많이 생각하고 있었다. 여러가지 이유가 있는데 오늘 몇 이유만 논할 것이다. 1. 분명히 한국어 문법은 영어에 비해 너무 많이 다른다. 영어는 “오른쪽으로 분지(分枝)의 언어"라고 하는데 한국어는 “왼쪽으로 분지의 언어"이다. 뜻이 무엇이나요? 예를 보면 이해할 수 있을 것이다. 간단한 문장만 말하면 (외국어를 말하는 남들은 간단한 문장의 수준을 지낼 수가 약간 드물다), 간단한 걸 기억해야 돼: 영어는 “SVO”인데 한국어는 “SOV”이다. “I’m going to school”라고 한국어로는 “저는 학교에 가요"라고 말한다. 영어로 똑바로 번역하면 “I’m school to go”이다. 두 언어 다르는 게 목적어와 동사의 곳을 교환해야 한다. 별로 어렵지 않다. 하지만, 조금 더 어렵게 만들자. “I went to the restaurant that we ate at last week.” 한국어로는 “전 우리 지난 주에 갔던 식당에 또 갔어요"라고 말한다. 영어로 똑바로 번역하면 “I we last week went to restaurant to again went”말이다. 한국어가 왼쪽으로 분지 언어라서 문장 중에 왼쪽으로 확대한다! 이렇게 좀 더 쉽게 볼 수 있다: “전 (우리 지난 주에 갔던 식당)에 또 갔어요”. 주제가 “전"이고 동사가 “갔다"이고 목적어가 “우리 지난 주에 갔던 식당"이다. 영어 문장은 오른쪽으로 확대한다: I (S) went (V) to (the restaurant (that we went to (last week))) (O). 그래서 두 숙어 문장 만들고 싶으면 생각속에서도 순서를 변해야 된다. 2. 첫 째 점이니까 다른 사람을 자기 말을 아라들게 하고 싶으면, 충분히

10 other things South Korea does better than anywhere else

Recently this article about 10 things that South Korea does better than anywhere else  has been making the rounds on social media, but when I first read it, I couldn't tell if it was sincere or satire. A few of the items on the list are not very positive, such as "overworking" and "using credit cards". So, I thought I would try to put together a better list. Here are 10 other things South Korea does better than anywhere else: 1) Small side dishes, a.k.a. " banchan " (반찬) Banchan are by far my favorite aspect of Korean cuisine. Rather than the "appetizer and main dish" approach of the West, a Korean meal is essentially built around small dishes. Even a 5,000 won (about $5 USD) meal at a mall food court will come with two to four banchan in addition to the "main", and often people will actually choose restaurants based  on the banchan (e.g., seolleongtang , or beef bone broth soup, places tend to have the tastiest kimchi). Ther

The King's Speech (and me)

Tonight, I finally gathered the courage to watch The King's Speech . Why did I need courage to watch a movie, you might ask? The reason is both simple and intricately complex: I'm a stutterer (Edit: person who stutters; "stutterer" is not who I am, but something that I do from time to time), and I have been for as long as I remember. Well, there it is - I've said it. To be fair, I actually don't remember stuttering when I was little. My first very distinct memory of stuttering was sometime in seventh grade, when I had trouble saying "nosotros" (we/us) in Spanish class. But I also remember knowing I was going to have trouble saying it, because we were going around the room, and I counted ahead to see what I was going to have to say. Which means by that point I was already stuttering. When did it start? That's a question for another day. So why am I publicizing this fact now? First, I'm in the midst of a lifelong attempt to "cure&quo