“Why would an anthropologist study unmanned systems?” This is a question I am often asked by engineers and product managers at conferences. The presumption is that unmanned systems (a reigning term in the field, albeit unreflexively gendered) are just that, free of humans; why would someone who studies humans take this as their object of study? Of course, we, as ethnographers, know there are always humans to be found. Moreover, few if any current systems are truly “unmanned” or “autonomous.”  All require human planning, design and maintenance. Most involve the collaboration between human and machine, although the role of the human is often obscured. When we examine autonomous systems (or any of the other terms invoked in the related word cloud: unmanned, artificially intelligent, smart, robotic, etc) we must look not to the erasures of the human, but to the ways in which we, as humans, are newly implicated.
My dissertation research, as well as research conducted with the Intelligence and Autonomy Initiative at Data & Society, has examined precisely what gets obscured when we call something, “unmanned” or “autonomous.” I’ve been increasingly interested in the conditions and consequences for how human work and skill become differently valued in these kinds of highly automated and autonomous systems. In this post, Tricia has asked me to share some of the research I’ve been working on around the role of humans in autonomous systems and to work through some of the consequences for how we think about cooperation, responsibility and accountability.
The Driver or the System?
Let me start with a story: I was returning to New York from a robot law conference in Miami. I ordered a Lyft to take me to the Miami airport, selecting the address that first populated the destination field when I typed the phrase “airport Miami” into the Lyft app. The car arrived. I put my suitcase in the trunk. I think the driver and I exchanged hellos–or at the very least, a nod and a smile. We drove off, and I promptly fell asleep. (It had been a long week of conferencing!) I woke up as we were circling an exit off the highway, in a location that looked distinctly not like the entrance to a major airport. I asked if this was the right way to the airport. He shrugged, and I soon put together that he did not speak any English. I speak passable Spanish, and again asked if we were going to the right place. He responded that he thought so. Maybe it was a back way? We were indeed at the airport, but not on the commercial side. As he drove on, I looked nervously at the map on my phone.