It Thinks, Therefore . . . ?

In “The Measure of a Man,” the best episode of Star Trek: The Next Generation, the only android in Starfleet, Data, is threatened by a human scientist, Bruce Maddox, who wishes to dismantle him, study his component parts, and create a myriad of duplicates. The weary Data insists he has a right to self-determination; the ambitious Maddox insists he’s a mere machine. A trial ensues. Captain Picard, representing Data, argues persuasively that manufacturing untold numbers of disenfranchised androids would establish a slave state. Indeed, the outcome of the trial will determine not only the singular Data’s rights, but the rights of an entire new race modelled after him. Dramatically, Picard asks Maddox, “Won’t we be judged by how we treat that race?”

As the philosopher Mark Kingwell contends in his latest book, Singular Creatures, the matter of human-robot relations must now move beyond the invented worlds of science fiction. We must explore the real-world ethical-political issues surrounding what Kingwell calls generalized artificial intelligences with autonomy, or GAIAs. Though the technology has yet to materialize, it already exists in our “speculative frame.” We had best prepare for its arrival through deep reflection on matters of metaphysics, personhood, and sentience, since we cannot embrace a “posthuman” world until we have confronted “the existential contours of the bare human condition.”

Rather than a dry technological study, Singular Creatures offers a timely meditation on two interrelated problems of philosophy: human consciousness attempting its own self-understanding and human society attempting to quantify what constitutes “conscious life.” References to thinkers like Aristotle, Heidegger, and Marx abound as Kingwell deftly argues for the urgency of these conceptual debates. Only negotiation of such dilemmas can combat the existential worry that will undoubtedly accompany what the computer scientist John von Neumann has dubbed the Singularity: the moment at which GAIAs emerge that are capable of producing improved versions of themselves — maybe even at a rate uncontrollable by humans.

Kingwell explores four hypothetical scenarios in which the Singularity may occur. First, GAIAs could emerge as “wholly owned property,” such as driverless cars and military drones, effectively slaves without personhood status even though they possess abilities far beyond those of their human creators. Or perhaps they are destined to become “welcome semi-individuals,” like pets or children, granted affection and respect but not freedom or independence. We may go so far as to recognize their “full autonomy even under conditions of radical otherness” (the aforementioned Data, along with the Cylons from Battlestar Galactica, model this result). Finally, we may see the emergence of “posthuman hybrids”: tech conjoined with our familiar biology.

As the theorist Donna Haraway has argued, this fourth scenario, in a sense, has already come to pass. We are all already cyborgs: think of our eyeglasses, phones, watches. And we are destined to become more bionic over time. Such unions could even represent our next evolutionary step, as posthuman transformations become increasingly intricate, powerful, and — perhaps — permanent.

Notwithstanding the nature of its emergence, what will sentient AI mean for the humans caught in its wake? There’s no clear answer, of course, but Kingwell makes the case for studying the humanities as a means of speculating on — and coping with — the eventuality. Literature, history, art, music, and the like already teach us about ourselves; indeed, they might also help us relate to our future robotic brethren, because the robots could be more human than we expect.

In his most thought-provoking section, Kingwell contends that any sufficiently intelligent robot collective will eventually demand social justice. After all, nobody likes to be exploited, whether they are “cloned, built, or born.” This train of thought inspires any number of questions. If GAIAs cannot “die,” can they be employed indefinitely? Can they acquire and transfer wealth? Should they be permitted to reproduce unlimited numbers of immortal offspring? Such issues are not easily resolved within our current ethical and legal frameworks.

And what of the age-old fantasy that a robotic labour force will liberate humankind from grunt work, allowing us to pursue our passions endlessly? Kingwell suggests that it’s not so simple. Our current stage of social development is one of “techno-capitalist immersion.” We cannot conceptualize technology as distinct from the structural forces of the market that allow for its creation, distribution, sale, and planned obsolescence. Accordingly, we can hardly imagine a post-work world except insofar as it persists as a “nightmare of meaninglessness.” Without tasks to structure our sense of purpose, we inevitably ask ourselves, is there any point to human life at all? If we don’t alter this perspective, surrendering various forms of labour to robots may inspire a human downgrading rather than the machine uprising foreseen in films like I, Robot and The Terminator. If we do get past it, perhaps we may orient the meanings of our lives in ways presently unthinkable, as freedom from work broadens the horizon for the human imagination writ large and restructures how and what we consider “achievement” and “responsibility,” among other experiential concepts. Admittedly, Kingwell’s detour into these speculations somewhat derails the book’s focus; readers are offered an extended monologue on capitalism, phenomenology, and temporality, without much reflection on the robots.

Proceedings get back on track, though, when Kingwell makes a compelling case that posthumanism resembles other movements for social and political justice. After all, there isn’t just one way of “being” a person. The hypothetical advent of GAIAs provides an occasion to probe the metaphysical status of other non-human beings, like animals, and the moral status of marginalized peoples fighting for gender and racial equity, among other contemporary crusades. The possibility of the posthuman makes us confront our fear of otherness; it dispels our illusions of human exceptionalism and our assumptions about who (and what) can claim personhood, in both abstract and legal senses. For Kingwell, it leads to this logical conclusion: There has never been a pure human existence, a life unburdened by extrinsic developments, including technological advancement. We have been perpetually subject to boundary blurring and indeterminacy, and the contours of our normality have always shifted accordingly. In this sense, to be human is to be transhuman; change itself characterizes those existential contours of the human condition that he implores us to cross-examine.

To that end, one mustn’t dismiss criticism of technology by labelling it as an illogical fear of change. Not only does this attitude reinforce the archaic technophile/technophobe binary, it presumes technology is always positive. It venerates our mechanistic productions as a sort of reason incarnate. Countering this overly pro-tech position, Kingwell contends that reason can never be concretized in an external object, no matter how advanced, because the concept can never capture the entirety of one’s inner experience. (Or, in the words of the poet Margaret Avison, “Nobody stuffs the world in at your eyes.”) At one point, Kingwell recalls the title of a Francisco Goya aquatint, The Sleep of Reason Produces Monsters. Reason is equivalent to “sleep” because so much of what makes up personhood falls outside of reason’s purview — and neglecting the validity of such unreason turns person into monster. Hegel also took up this notion in The Phenomenology of Spirit, insisting that when we subscribe to a “wisdom in sleep,” our existence becomes non-human: “nothing but dreams.”

More to the point, it’s naive to suppose that technology is always under our control merely because it’s a “product of our emergent desires.” The things we make are never neutral; their impact ripples unpredictably across the surface of human society, like stones thrown into water. Our techno-capitalist world, which tends to lionize any advancement, big or small, is especially poised to suffer unanticipated ramifications. This isn’t to suggest we should embrace neo-Luddism. Rather, the meditation offered by works like Singular Creatures is an important stopgap against “progress” becoming a justification for human depravity, as it has been too often in our history. Not every advancement is good; not every advancement is bad. Clear-eyed reflection helps us differentiate between the two, so that, when the time comes, we may make an informed decision on whether to dismantle Data.