THE MYSTERIOUS GRAY AREA between cognitive science and computer science has long held broad allure, seducing thinkers at least since the time of Norbert Wiener and the theorization of cybernetics at midcentury. In the ensuing decades, what began as the technical field of artificial intelligence has increasingly captured the imagination of an emerging generation of prolific scholar-artists who are mining AI for new forms of expression and reception. Working primarily in institutional contexts characterized by descriptors such as new, integrated, and digital media, they are pursuing intensive practices modeled on scientific, social-scientific, and humanities research but probing an intriguing range of topics still only loosely defined, and presenting their findings through a range of innovative exhibition strategies. D. Fox Harrell, a leading representative of this trend, is a practicing artist, as well as an associate professor of digital media in the Comparative Media Studies Program and a researcher at the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology in Cambridge. His profoundly ambitious and wildly eclectic Phantasmal Media will likely find a wide audience among artists and technologists alike.
Harrell’s conventional scholarly production is serious and well respected, but it is his principled artistic practice, which probes the potential of new computer technologies to transform cultural production, that seems to have provided the conceptual foundation for this book. His GRIOT system, for example, which he began developing in 2002 and is named, with warm irony, after the West African storytellers famous as repositories of oral tradition, musical knowledge, and political thinking, is a “subjective artificial intelligence system” developed to produce poetry and imagery that can (he claims) narrate from multiple cultural perspectives. This is only possible, Harrell argues, because his program uses a series of algorithms and data structures to generate narrative based on user input in the form of specific keywords. The feedback between the code’s architecture and the user’s subjective choices ends up producing an ambiguous and potentially infinitely variable (Harrell prefers the term polymorphic) set of metaphors that nonetheless retain a core concern with issues of cultural identity. In The Griot Sings Haibun, 2005–, and Living Liberia Fabric, 2010—two works produced using the Griot system—a user contributes “equally” in dialogue with the “system” to the process of “blending” metaphors, producing a poem or series of poems in the former, and virtual textiles of imagery in the latter. The effect of reading or observing these “sample outputs” side by side is slightly surreal, as though one is looking at the results of a kind of literary-aesthetic Turing test. Harrell’s presentation of such works in Phantasmal Media is occasionally labored, as he seeks to define the novel formats explored in his practice. But one question recurs in the mind of the reader throughout: Where is the art—in the software, in the poems the software produces in dialogue with a human user, or in the mind of the user?
Harrell’s perspective on such questions is particularly valuable, since, as an artist himself, he takes art seriously—arguably more so than the devotees of “neuroaesthetics,” typically trained in art history or neuroscience, who are responsible for most current attempts to apply cognitive science to our understanding of art. In fact, Harrell’s book addresses two major limitations of the neuroaesthetics movement: first, the idea that there is something like a universal and normative human subject defined entirely by the physical structure of the brain; second, that the image under consideration by that subject exists as an a priori object, acting on the subject in knowable, predictable ways. Harrell rightly argues that the reality of subjective experience is much messier. He writes in sensitive and often quite moving ways about his critical engagement with issues of race, class, alienation, and social justice, much of it motivated by his own search for identity as a black man in a predominantly white academy and as a techno-artist who inhabits the worlds of science and art and the sometimes uncomfortable spaces in between.
It may seem tempting to characterize Harrell’s book as yet another in a long line of earnest but doomed efforts to stitch back together those “two cultures” of art and science that C. P. Snow famously argued were torn asunder by the Enlightenment. (Harrell’s position at MIT calls to mind the cross-cultural attempts of figures such as György Kepes and Nicholas Negroponte.) But to do so would be to miss both the core of Harrell’s project and the potential of the work that it may well set in motion. The book is an unabashed effort to produce a unified theory of media systems, one that can reground computation in the service of a cultural and political agenda.
Harrell’s fundamental goal is to change not just the way computational tools and media systems operate at a technical level but the ways in which we design and understand them. He advocates a method that considers not only typical objectives such as efficiency but a host of other cultural values, and characterizes this strategic rethinking of computation as “critical computing.” This approach, he argues, has the potential to reconfigure the systematic (but as yet not monolithic) relations between human beings and computers, thereby enhancing human “freedom” while combating a host of “social ills”—everything from “violence [and] atrocities of war” to “small acts of unkindness [and] prejudice.”
This means that Harrell’s book is not only a piece of well-intended humanism; it is also unquestionably a manifesto—a manifesto about ghosts. Phantasms, as Harrell redefines the term, are figments of the imagination that nonetheless play a critical role in structuring lived experience—the social signs, preconceptions, and cultural assumptions that influence everything from casual social interactions to international relations. They are “combinations of mental imagery and ideology constructed by embodied, distributed, and situated cognitive processes.” Phantasmal media, in turn (both sequentially and tautologically), are “media systems that prompt phantasms.” With this in mind, Harrell argues—twisting and turning through expositions of cognitive scientific models of thought and action, in dialogue with critiques of racist and colonialist ideologies—it becomes possible to refocus the design of AI software to construct and deconstruct specifically targeted phantasms, “revealing” particular illusions and misconceptions about everything from politics to identity, which might otherwise remain invisible, “backstage” in the human mind.
Harrell opposes examples of the “semivisible shaping of people’s shared phantasms”—so-called recommender systems, Amazon’s “Recommended for You” being a prime example—to his own work in AI, which seeks to pose questions rather than offer (artificially constructed) answers. But there is a thin line between the behaviorism of Amazon and the positive phantasms of Harrell. There is nothing fundamentally less “semivisible” about the AI at the core of Harrell’s ambiguous artworks; the main distinction lies in his generous efforts to explicate his methods.
Harrell’s intellectual and institutional position means that he can take what he likes from various natural (and artificial) scientific literatures while leavening this material with readings gleaned from history, anthropology, art history, postcolonial and race theory, literary criticism, and much else besides. His range of reference will appear astonishing to many—one spread cites Mikhail Bakhtin, George Herbert Mead, Herbert Blumer, Michel Foucault, Roland Barthes, and Brad Gallaway in rapid succession, and on the following page, a block quotation from Kafka’s “The Metamorphosis” adds to the cacophony. But readers hailing from a more defined disciplinary training may find Harrell’s syncretism merely idiosyncratic and baffling. At times, Harrell himself forthrightly acknowledges that the wealth of reference on display here does not necessarily promote a synthetic understanding of the amorphous techno-aesthetics under discussion.
Such a barrage makes for interesting reading, and Harrell’s measured presentation, in alternating chapters, of theoretical frameworks for understanding cognition and aesthetics and explications of his own artistic and technical experiments somewhat mitigates the otherwise confusing proliferation of citations. Even if issues raised in one chapter are rarely addressed in the next, the juxtaposition of scholarship and artwork here does convince the reader of the necessity of asking difficult questions about computation, and of asking them often. Harrell concludes by emphasizing that it is crucial to remember that “all computation is composed of mere smoky, subjective phantasms and all meaning, including computationally prompted meaning, is ultimately human imagination.”
But it is important, too, to ask about the specific value of Harrell’s terminology and method. Most urgently: What is meaning? This is a charged question in both media studies and art history, but Harrell leaves it unasked. Despite the fact that this word appears more than a hundred times in the book and is one of his central concepts—the primary result desired from human-machine interaction—it does not receive definition(s) or even genuine qualification. We are told that it should be “rich” and “expressive,” but expressive of what? Readers interested in the finer points of hermeneutics and so-called posthermeneutic media archaeologies will not find these topics discussed here. And certainly it is worth asking whether any media, digital or otherwise, exist that do not prompt the creation of phantasms. These specters may well be as old as technologies themselves, as suggested by the many scholarly studies that identify ghosts in the (literally mechanical) machine, such as Friedrich Kittler’s landmark Gramophone, Film, Typewriter (1986).
Then there is the question of the relationship of “phantasm” to “ideology.” Harrell argues that the distinguishing characteristic of phantasms is that, unlike ideologies (which he seems to understand sometimes as mere political attitudes, at other times as “worldviews,” and at still others as simply equivalent to “ideas”), they are produced through the “conceptual blending” of knowledge and belief with images. But ideology has always been defined by image and imagination—the term was crafted by A. L. C. Destutt de Tracy in 1796 from the Greek eidos (image) and logos (word, concept). The shadows on the wall of Plato’s cave were already images of ideas and ideas of images, and one suspects that the proliferation of neologisms and jargon in Harrell’s book (which includes a wholly necessary six-page glossary packed with terms such as semiotically socketed social-computational flow), reveals him to be shadowboxing.
At the end of his preface, Harrell courageously admits that the ambition of his project may also be its undoing: “From one perspective, I submit this book to you as a wistful, probably doomed, love letter—hoping for an ‘I love you too’ (your theories make sense, and open up new possibilities), but I am prepared for my ideas to receive the fate of all words of unrequited lovers. That is, even if it ends up not laying a blueprint for the future, I strive for this book to be good poetry.” It is, of course, far too soon to judge, but Phantasmal Media will unquestionably provoke necessary thought and debate along with the passionate embraces and derisive dismissals characteristic of the often overheated atmosphere of digital-media discourse. So even if it isn’t good poetry, it’s good for all of the many fields of creative and critical endeavor that it addresses.
John Harwood is an associate professor of modern and contemporary architectural history at Oberlin College in Ohio.