Brain Parade When Robots Kill
Posted by Jose on Thursday, 29 of June , 2006 at 5:43 am
We get a bit more serious today as we turn out attention to military robotics. First the question:
The military is increasingly using robotic technology. What kinds of ethical considerations should we be making before we automate killing?
MT: For my take I first have to state that I see the trend of increasingly autonomous weapons as nigh unstoppable. So I’m not going to whinge about should we or shouldn’t we as that is tantamount to pissing in the wind. However what does concern me is oversight. One of the advantage of human soldiers is that they’re social animals, they relate, talk to the press, leak stories, etc. The military can be notoriously secretive but soldiers aren’t, eventually all kinds of stories leak. Computers on the other hand don’t get drunk in bars and spill the beans to journalists. Future robotic weapons (especialy flying ones) will probably be quite stealthy to boot What I’m worried about is a future where our primary insight into what our militaries get up to comes from military PR reps.
This is very worrying as its theoreticaly down to us as civilians who have the ultimate say in the kinds of things our militaries are doing. That might be hard to do if we have no oversight over, or knowledge of, the machines killing in our name.
I’ve also lined up commentary on this one from science fiction authors John Shirley and Robert Charles Wilson, Roboticist Daniel Wilson, tech blogger Jan-Willems Bats, science blogger Alan Bellows, anthropologist Dr. Daniela Cerqui and technoprogressive Dale Carrico.
Dr Daniela Cerqui
I would like to reply with another question: why should automation be questionable just in the military field? Of course, we never mention ethical questions when we feel that automation is not hurting anyone, but we should be aware that, fundamentally, the logic is the same. Technology is never neutral; our values are embedded in it. We are externalizing our human qualities in the robots, we are even trying to reproduce life, as if we were gods. And this might lead us to the end of humankind, even without automated killing! There are many people thinking that human beings are just one step in the evolution, that complexity must increase and that
we will naturally disappear to let place to a more complex species. But contrary to all the other extinct species, humans are planning their own extinction, and that sounds very strange to me, as an anthropologist.
Dr Daniela Cerqui is a social and cultural anthropologist currently conducting research at the Department of Cybernetics of the University of Reading
Alan Bellows:
Strictly speaking, lethal robots have been part of warfare for centuries in the form of booby traps and landmines. These automated killers have taken countless lives, discriminating solely on whether the target is aware of the device. But the robots of tomorrow are not intended to lie in wait, rather they will seek out targets proactively, and with more refined discrimination.
Despite the fact that most people are repelled by the idea of a killing machine, a sufficiently advanced robot would make fewer mistakes than a human, and result in fewer unnecessary deaths on average (less friendly fire, fewer civilian casualties, etc). In fact, one could argue that it makes more sense to leave life-or-death decisions to the impartial logic of a robot’s microprocessor rather than putting that responsibility into the hands of mistake-prone, inconsistent, biased human soldiers.
Ultimately, the question is whether it is ever ethical for a robot to take a human life. In order for a robot to judge whether a subject should be killed, human operators must first assemble a list of characteristics which the robot can use to qualify a target for lethal force. No such list can be perfect, which means that designers must balance the loss of expensive equipment (err on the side of caution) with loss of life (murder liberally). This balance between the ethical and the practical leaves a gray area within which no point is completely acceptable to everyone.
There is also the very real concern that a military which can go into battle with negligible human losses is more likely to make war. When a robotic army can be unleashed upon any disagreeable region in order to force change, the owner of that army will wield an incredible power. One hopes that the world’s militaries will concentrate their resources developing robots whose goals are more useful than mere killing, such as medical attention, sabotage, surveillance, and non-lethal incapactiation of enemies. Such robots would help advance military goals without being roving murder machines.
But it is easy to look into the future and question the ethics from our safe, objective perch in the present. One day we will have the technology to build such robots, and when that day comes, the question we will face is whether it is ethical to risk the lives of a dozen soldiers to do a job which could be accomplished by a single weaponized robot. From that perspective, I think most would people would opt to send in the Death-o-matic.
Alan Bellows is the designer and writer who founded Damn Interesting
John Shirley
It’s a good question–I think that horse got out of the barn when we started using ’smart missiles’ and jumping up and down with glee when we could watch them home in on their targets. The upside is that we risk less of our soldiers, but the downside is that a robot or drone etc cannot easily distinguish between civilians and adversaries. But you know we already seem indifferent to that concern–we have taken to dropping bombs on houses that might have terrorists or insurgents in them, maybe, not caring that they certainly have women and children in them too. We should be sending in commandos to these places, not bombing them just in case.
John Shirley is a science fiction author whose novels include City Come A-Walking and A Splendid Chaos . John has also fronted punk bands and written songs for the Blue Oyster Cult.
Robert Charles Wilson
The one I try to keep in mind is that war and all its shiny chrome and leather paraphernalia, from siege engines to standing armies to robot drones, is never more than — at best, and rarely — a necessary evil. It may, like dental surgery, sometimes be unavoidable, but it isn’t glorious nor is it morally redeeming. We’ve lived all our lives a world where a button can be pushed that would destroy multiple millions of human beings within minutes, so “automated killing” hardly seems like something new or astonishing. These questions ought to have been addressed long before Hiroshima.
Robert Charles Wilson is the author of this years Hugo nominated Spin and Blind Lake
Daniel H. Wilson
This is a tough question, so I’m not going to answer it. However, it bears mentioning that the majority of military robots play support roles. Unmanned aerial vehicles (e.g., the Predator drones) fly around ferrying communications and collecting information — and very occasionally shooting people. Backpack-sized robots such as iRobot’s Packbot are used by infantry to sniff out improvised explosive devices (IEDs). Newer legged prototypes, like the impressive diesel-
powered BigDog from Boston Dynamics, are able to run around on four
legs carrying supplies.
Daniel H. Wilson is the author of How to Survive a Robot Uprising
Jan-Willem Bats:
Automating the military will lower the number of human casualties that war inevitable demands.
Imagine a world where everybody’s army is 100% robotic. If two countries were to wage war, they would basically be playing a real-time-strategy game (such as Command & Conquer) against each other, only in real life. The one with the better technology would inevitably win.
The good news is that human casualties will have been kept down to an absolute minimum. Some innocent bystanders may have gotten killed, but genocide would be avoided.
The bad news is that countries may go into war more easily. Think about it. Robots can share their programs, so you can easily upload one program to a thousand robotic soldiers. .. robotic soldiers that are cheap and expendable… robotic soldiers that require no food and no sleep.
We know that the best technology always wins. We also know that power corrupts. Will the world eventually be taken over by the army with the best technology?
Jan-Willem Bats is the computer obsessed blogger behind Our Technological Future
Dale Carrico
Well, I think ethical considerations should compel us to reject the automation of killing altogether. Ethics also has something to say about the social costs of the war addiction of our bomb-building elites, and about the long-term personal and social costs imposed by the brutal roboticizing process that transforms citizens into soldiers in the first place. You know, killing a human being should simply never seem easy. It’s so obvious it sounds sanctimonious to point it out, but there it is. And since we’re having this exchange in a time of war it should be said often and loudly as well that definitely we know we’re in trouble when so many of our elected representatives sound glib at best when they say war is a last resort. Every war is a disaster, every war is a defeat — even when we “win†one. Wars of choice like the current catastrophic Iraq adventure especially bespeak an almost unfathomably profound breakdown of the ethical imaginary.
The automation of mass violence — via mass media distraction, via the video-gamization of weaponry, via the neuroceutical modification of soldiery — is an extraordinary intensification of the techniques of training and drill that have long functioned as a ritual instrumentalization of the individual soldier. This instrumentalization has everything to do with the obliteration of ethics in the encounter of subjects in a war-zone and its replacement with an encounter between objectified no-longer-quite subjects. The outright roboticization of militarism is a step along a tragic trajectory rather than the appearance of something altogether new.
Dale Carrico is a lecturer with Department of Rhetoric at UC Berkeley, a moderator of the Technoliberation group and he also maintains this blog.
Related
Follow up on Amor Mundi (link)
Mempunks discuss the Crusher (link)
Related posts:
Category: Science Brain Parades, Philosophy and Ethics, Technology, Predictions, Brain Parades
Tags:Brain Parades, Musings, Philosophy and Ethics, Predictions, Science Brain Parades, Technology
Subscribe to Comments: RSS
Pages: [1] 2 » Show All
2006-06-29 08:31:00
To be fair I am I no way qualifies to answer that question, but it is an interesting one, what scares me is the thought of governments just pulling out their remote control device like a kid with an R.C. Car and driving it straight into someone elses. The problem is whats to stop any rich country going to war, in modern society countries have gone to war even though population didn’t want it to, (sure you can think of one example there) but I would hope that for the most part at least. If some evil dictator came into power there would be some kind of uprising or refusal to fight if the means were extremely unjust. With an automated army however there is no such opinion.
preston parris
btw can normal people get involved in your brain parades?
2006-06-29 08:43:00
I’d say you just did.
I’m not a snob when it comes to the Brain Parades. I’ll invite whoever I think might have something interesting to say. However I only invite people whose writing I’m already familiar with either through their books, their blogs or our comment section.
2006-06-29 09:13:00
cool, well i will try to keep up with the comments and mayeb one day ill get an invite
love the site btw, keep up the good work
2006-06-29 10:44:00
In a “military SF” novel I’m writing now, I predict that robotic weapons are a dead end.
Dead. End.
Why? Simple. Countermeasures. EMP weapons.
A revolver still works after an electromagnetic pulse has made all the drones drop out of the sky. And the gun is held by — surprise! — a human being.
If the U.S. military is so unbelievably naive as to think robots are unstoppable, they are setting themselves up for a BLACKHAWK DOWN-style defeat — only much, much worse.
2006-06-29 17:41:00
One of the advantage of human soldiers is that they’re social animals, they relate, talk to the press, leak stories, etc. The military can be notoriously secretive but soldiers aren’t, eventually all kinds of stories leak. Computers on the other hand don’t get drunk in bars and spill the beans to journalists.
There are still - and always will be - people in the link somewhere. They may not be the trigger pullers but the bots will need refueling, rearming, programming, someone to fly them to the theatre or launch them, and so on.
A.R.Yngve In a “military SF” novel I’m writing now, I predict that robotic weapons are a dead end.
Dead. End.
Why? Simple. Countermeasures. EMP weapons.
When I was in the Marines (late 80s) EMP was not a big deal but all the milspec electronic equipment we had was hardened against EMP. EMP is a big deal but I can’t think it won’t occur to people to harden the new bots against the effect. Likewise countermeasures - we have to worry about that now with the manned equipment.
Dale Carrico Well, I think ethical considerations should compel us to reject the automation of killing altogether. Ethics also has something to say about the social costs of the war addiction of our bomb-building elites, and about the long-term personal and social costs imposed by the brutal roboticizing process that transforms citizens into soldiers in the first place.
We’re a few thousand years too late for that. The evolution of weapons from stick to spear to sword to machine-gun is all about making it more effecient. I can’t see a world where we lay down rifles for rocks, which is where your thoughts would lead us.
Nice to think about, just isn’t going to happen.
2006-06-30 01:23:00
a. r. yngve got it right, but for completely the wrong reasons.
Nothing to fear from robot weaponry - there’s going to be no oil to run them. Don’t tell me that we’ll develop alternative sources of high-energy-density fuels, etc. - there ain’t enought time. Expect rolling blackouts to begin being a permanent feature of life across the globe starting in about 2008 (in fact, one might argue they’ve started already: http://www.google.com/search?q=%22cape+town%22+blackout)
and by 2012 we can expect serious social disruption, starvation, etc. as energy shortages unravel the globally interlocked economies, causing massive food, material and manufacturing shortages. (http://en.wikipedia.org/wiki/Olduvai_theory)
Good luck with the robots. I’m more concerned with How to Make Fire.
2006-06-30 06:45:00
Brian,
Your comments about bot maintenance staff is well taken. However those people are much fewer in number and probably stationed in highly secure facilities. They don’t seem to be leaking stories to the press now and I doubt this situation will be changed in the future.
I agree that its too much to expect militaries to abandon these technologies. Their implementation is inevitable. What I think we should be looking at is ways we can ensure that civilian oversight of our militaries remains rigorous.
2006-06-30 11:17:00
This post has been removed by the author.
2006-06-30 11:27:00
Personally, I’m not particularly worried that a robot army is going to march over the horizon any time soon. The logistics of operating and maintaining ground based robots mean that their main advantage is simply reducing the risk to human personnel on the front line. You still need lots of skilled, expensive and emotive individuals nearby, so all your opfor has to do is refuse to engage your robot troopers, and pick on the soft targets of the support crews. Robot technology is going to have to advance a long way before the infantryman can be replaced.
UAVs are a completely different kettle of fish, in that the logistics of keeping them running are more or less the same as those of keeping a conventional aircraft operational. Even the network systems needed to tranfer data to and from remote vehicles has been around for a while, in the form of JSTARS. In fact, advances in combat UAVs have come so fast that its beginning to look as if the next generation of combat aircraft (Typhoon, F22, JSF etc.) is going to be obsolete before they even enter service.
This, to me, is a far more worrying development, as it dramatically reduces the costs of operating the most potent convetional weapon systems, to the point where, for example, the CIA can carry out its own airstrikes without even needing the cooperation of the military. And I don’t know about you, but to me the thought of bodies even more secretive and unaccountable than the regular military having access to that kind of firepower is just a teensy bit frightening.
2006-06-30 11:58:00
Remember when someone figured out you could wreck the SDI’s orbital laser-mirrors with a bucket of sand?
I’ve been working in the IT industry for over 10 years. And the more I learn about computers, the more I realize how vulnerable and dependent on humans computers in general are.
Consider this: even if MILITARY computer systems are well shielded against EMPs and potential electromagnetic weapons, the CIVILIAN infrastructure which supports the military is not.
And if an enemy can’t afford the expensive robots, he will invest in the relatively cheaper and simpler means of sabotaging them.
Engineers tend to overlook the ingenuity with which humans can sabotage fragile machines. I unserstand why: if you’ve worked hard to design an impressive device, you don’t want to think of the many ways in which humans might break it.