TWO HUNDRED AND SEVENTY-SEVEN PAGES, column after column of digits so tiny and so densely packed that, even with my reading glasses on, I have to use a magnifying glass. Three sets of numbers in each column, eleven columns on each page. This is the principal part of Florian Hecker: Halluzination, Perspektive, Synthese, the catalogue produced in the aftermath of sound artist Florian Hecker’s 2017 exhibition at Vienna’s Kunsthalle Wien. Leafing through the volume, I am lost in what feels like an endless, undulating expanse of ciphers I cannot possibly hope to make sense of, abstractions more brutal than most conceptions of modern art could ever have prepared me for.
Yet while numbers as such are abstract, numbers relating to concrete things are not: They are part of the world of actual phenomena. And the endless columns of numerals are exactly that. They express the values and measures of “timbre”—an elusive quality of auditory texture whose significance is largely founded on the fact that, unlike “pitch” or “harmony,” it cannot be exactly described. Timbre could be the overtones you hear as a sound lingers. It could be the warmth of a particular violin or the sudden shrill edge of an angry voice. There are so many ways of hitting a high C—and timbre is, essentially, what differentiates them. No wonder that a modern age focused on scientifically capturing, calculating, and reproducing even the most fleeting sensations would become fixated on it. In modern music, composers such as Wagner and Debussy were obsessed with timbre, attempting to foreground and celebrate its complexities in their compositions. Others endeavored to synthetically produce it: In the field of electroacoustic and electronic music, composers such as Pierre Schaeffer, Karlheinz Stockhausen, and Iannis Xenakis sought to both modulate and replicate preexisting sound textures—and create new ones from scratch. But taking seriously the very elusiveness of timbre—that it is essentially a hybrid signature of a concatenation of events including not just the vagaries of the sound source but also the ear and brain of the individual listener—how do you actually produce it?
THE 277 PAGES of numerical values attest to a radical effort to not just overcome this challenge but keep the question of timbre as such alive. The pages are, to put things simply, the printout of a computational process that replicates, at the machine level, the very process of listening. The ciphers represent the “inner sound” heard by a machine and might in principle be used to regenerate such hearing. The key context here—the field of research in which Hecker’s work intervenes—is the crossroads between neurology and computer music. In the mid-1990s, researchers managed to create an exhaustive map of mammals’ primary auditory cortex, connecting each individual neuron to the specific auditory stimuli—the so-called time-frequency representation patterns—that would maximally excite it. At this stage, no attempt was made to invert the process. If you knew the exact response patterns of specific neurons, you could theoretically reverse engineer a sound from the perspective of neurons that had been exposed to it—that is, synthetically re-create the sound as constructed by the brain in hearing. Such listening to the very act of sensing was only made possible when machine-listening researchers Vincent Lostanlen and Joakim Andén developed what they called “time-frequency scattering”—a form of signal synthesis that is comparable to the training of a deep neural network and that approximates the brain’s multifaceted encoding of sound as sensation. The “scattering” was, in fact, a reference to a term used in quantum mechanics for microscopic phenomena such as the shimmering of a pearl or the reddish shade of a sunset—evanescent effects that, like timbre, seem to involve a radiating maze of nonuniformities.
Lostanlen and Andén’s research was purely scientific; it wasn’t until Hecker heard about their work that it was appropriated for aesthetic purposes. In Hecker’s view, time-frequency scattering could create a new type of sound texture—a timbre that would effectively replicate the very event of listening, in this case by distinctly nonhuman ears. A collaboration ensued in which Hecker and Lostanlen repurposed time-frequency-scattering software not simply in the service of music production, in the more limited sense of the term, but to perform an explicit remediation of the intertwined problems of sound texture and synthetic creation that emerged at the turn of the twentieth century. Calling forth the fantasies of Stéphane Mallarmé, Claude Debussy, and Vaslav Nijinsky, among others, it was a remediation that would slip across the fields of the written and spoken word, dance, stage setting, and sound production to re-create, from the perspective of the early twenty-first century, the forces at work in the birth of artistic abstraction across the arts. And at the center of all this was a mythological creature, one that rarely visits the contemporary imagination: the faun.
TIME, IN OTHER WORDS, TO ZOOM OUT. One hundred and fifty years on, from within an expanding computational culture, we are invited to revisit what was once known as modern art—a phenomenon that was less a particular series of styles than a form of esthesis that, again, comes across as peculiarly acute, insistent, restless. From this contemporary vantage point, the endless bickering over the significance of artistic postmodernism and its distancing or undermining of modernist abstraction seems much like the contortions of a previous century, during which the very idea of modernism had grown threadbare, reduced to truisms that were increasingly associated with placid postwar optimism and prosperity. Pax Americana and so on. That particular attitude has now very definitely been exploded. New existential uncertainties—a full-blown ecocrisis, rising fascism, geopolitical instability, technological development run amok, with novel realms of sociality and sensation and forms of exploitation unfolding at unprecedented speed—place us in a situation that is in many ways analogous to that of the late nineteenth and early twentieth centuries. Numerous phenomena compel us, once again, to explore, reflect on, and respond to the new abstractions that surround us. Think, for instance, of what Matteo Pasquinelli calls the “metadata society”—a regime in which harvesting information about collective behavioral patterns has become a significant economic and political instrument in the effort to control the future. Or consider the attempts to simulate natural language by training artificial intelligences on exponentially big data sets: Prominent Google researcher Timnit Gebru claims she was fired for having pointed out that not only do data sets this size tend to scale up all sorts of social biases, they also turn language itself into a synthetic modeling of something that might look like meaning but that is removed from the contexts on which actual understanding depends.
Such abstractions, arising from new forms of synthesis, have material stakes: They reconfigure lived reality in concrete ways. And since they are already fully real, their power cannot simply be dispelled by reactionary appeals to the actual and tangible. Better to go on creating abstractions—to get on the inside of that process, so to speak. Which is, of course, why we may feel a certain pull toward the late-nineteenth-century moment when synthesis first became an industrial as well as an aesthetic phenomenon. And this is where the faun—a synthetic man-animal construct that gestures toward the modern inventions of synthetic smells, sounds, colors, and fabrics—comes in. Hecker’s FAVN, produced in 2016 for the Alte Oper in Frankfurt, is perhaps best described as a resynthesis of an original investment in synthetic creation: notably, a series of famous artistic works spun off from Mallarmé’s 1877 poem “L’après-midi d’un faune” (The Afternoon of a Faun). Mallarmé’s poem was an attempt to stage, in and through words, the very impossibility of grasping the elusive sensations produced by a faun’s dream vision of seductive nymphs. Debussy, in 1894, taking off from Mallarmé’s strings of metaphors swirling around an empty target, presented “Prélude à l’après-midi d’un faune” (Prelude to the Afternoon of a Faun), a symphonic poem in which sound texture comes to the fore and tonality and harmony slip away, as a very minimal melodic theme passes, fleetingly, from instrument to instrument. Eighteen years later, Nijinsky used Debussy’s “Prelude to the Afternoon of a Faun” as the musical score for his ballet version of Mallarmé’s poem, in which the dancers were made to perform strangely frozen, difficult, inorganic movements—shockingly—in bare feet.
The controversies generated by these turn-of-the-century works still echo in Hecker’s opera version. More brutal abstraction: Onstage in Frankfurt were no singers or dancers, just three objects—a syringe-like white column that doubled as a loudspeaker; a wood-framed folding screen with densely pleated white fabric; and a thin, leaf-green backdrop hanging freely in space. Before each of the opera’s three movements, a computer-generated voice emanated from the loudspeaker reading from a libretto written by philosopher Robin Mackay that contained fragments of text extracted from writings by Mallarmé, Debussy, and others. And the music? It was essentially extracted from nothing—or, more precisely, from an undifferentiated timbral field produced by a machine-learning, or listening, procedure known as “wavelet scattering.” Wavelets are analytic tools: a means of describing sound signals or data that is increasingly replacing Fourier analysis, which has been key to sound reproduction and synthesis since the nineteenth century. For while Fourier analysis translates sound continuities into sine and cosine waves, wavelets facilitate a granular approach, reading sound as a number of discontinuous features that never need to be assembled. This is why Hecker could repurpose wavelets as the synthetic counterparts to the truly multifaceted way in which the sound is sensed by the individual neuron. The sound of hearing is notably elusive, and an individual wavelet—a brief oscillation with an amplitude that starts at and returns to zero after a brief increase—is not something you can actually listen to. But wavelets can be scaled up: The swooshes, swishes, and swaths of white noise that started off the first movement of FAVN were the sounds of a vast multitude of wavelets listening to an initial signal provided by Hecker.
However, from that point onward, the entire timbral field kept being reprocessed or relearned—listening to itself in ever newer versions. And out of this recursive procedure, identifiable signals—individuated sounds—started to appear in bits and pieces. By the second movement, one could hear distinct patterns, thanks to the use of sieving algorithms that filter sounds at the granular level, eliminating sonic material like a sculptor chipping away at a block of marble. By the third movement, these sound patterns had gained in force, complexity, and dimensionality, confidently taking on space. You could literally feel the way they hit your ear. Aural hallucinations ensued.
SYNTHESIS ALL THE WAY DOWN, in other words—elusive sensation ceaselessly reworking sensation as this neural-network drama unfolds, ultimately enlisting the listener’s brain in the process. Through it all, however, the objects onstage remain still, processing the birth of modern art in their own manner. For just as in Mallarmé’s text, the faun and the nymphs are at once definitely there—and perhaps not. And anyway, it is as hard to tell these characters apart as it is to separate subject and sensation. The folding screen, with its pleated fabric, recalls the stiff, angular, nonhuman twists and turns of Nijinsky’s faun, but also the Grecian-style dresses worn by the nymphs in the same production. The tall loudspeaker column might call forth Mallarmé’s speaking faun, but its syringe-like shape evokes the “syrinx” mentioned in the poem—the name of the nymph that, in Greek mythology, transforms into hollow water reeds that are then made into a flute by Pan. And while the light-green fabric backdrop is an abstract, monochrome reminder of Léon Bakst’s verdant, expressionist set design, it also recalls the flimsy, undulating, divide between fleeting sensation and tangible reality in Mallarmé’s poem. Stillness does not, in other words, signal that identities are clear-cut or a work that has found its final, definite form. The work of abstraction is never done. Emerging from within the realm of machine sensation, the sensing-learning process goes on: The pages of numerical values attest, specifically, to the resynthesis of the entire sound production of FAVN for the Kunsthalle Wien exhibition. By the same token, the catalogue makes FAVN available for further resynthesis by anyone capable of using this information.
Ultimately, though, this confrontation with abstraction indicates a radical reorientation of a phenomenon that has been key to modern music: the fascination with acousmatics—sounds that have no identifiable, visible source. A legacy of the age of sound recording and the signature strategy of musique concrète and related phenomena, acousmatics is also an important feature of modern dance music. As disco increasingly emphasized the dance floor over rock’s cult of personality, anonymous disembodied voices for hire became an attractive practical alternative. It was the first step toward contemporary websites that offer electronic musicians “unlimited vocal downloads” or “1,000 free female vocal samples”—voices whose fictional names (Cooley, D’Layna) denote various pitch ranges, styles, and timbres, like the names and numbers of a Pantone color chart.
Psychoanalytic theory, of course, had things to say about disembodied voices as objects of desire and may have been vindicated by their proliferation in club culture. However, Hecker turns this particular understanding of acousmatics on its head. The sounds he summons forth may well appear out of nothing, having no identifiable point of origin or source. Yet disembodied is hardly the word you would use to describe them. His work, rather, assumes a very different understanding of bodies and embodiment—especially in those pieces in which Hecker explores not just sound in general but, more specifically, the realm of voices. Seen or not, voices carry information about the coordinates of interhuman space. As demonstrated in the binaural sound experiments of music psychologist Diana Deutsch, such information is also an effect of the ear of the listener. A recording of repeated two-word sequences, played on left and right channels slightly out of phase, resulted in the listener hearing words that were not in fact spoken—“phantom words,” different for everyone. In Word-Gammatone Muse, 2012, Hecker repeated Deutsch’s demonstration, but with the important difference that this time a third, nonhuman ear—a gammatone filter, often used as a digital approximation of the filtering produced in human hearing—was slipped into the production, completely changing the framework of the experiment. The result is a sound that feels at once crowded and multidimensional, disrupting the binaural sense of space in Deutsch’s work. Listening for hallucinated voices is now complicated by the sounds produced by an artificial hearing system. You are at once inside your own psychoacoustic space and inside a machine’s approximation of one. You, in short, have become a new type of body: a chimeric, synthetic creature. A faun, for example.
BY THE SAME TOKEN, you have also joined the new choir of voices populating our everyday: the Alexas and Siris, the nagging household appliances. What does the fridge say? In his 2010 piece GreenScreenRefrigeratorAction, Mark Leckey gave voice to a Black Samsung smart fridge, allowing it to reflect on its place in the world of shiny new appliances. In 2011, Hecker resynthesized the autotune voice track from Leckey’s performance, mixing it with his own 3 Channel Chronics, a 2010 installation in which visitors’ movements altered the sounds from three loudspeakers placed in the ceiling. The result was another impossible sound body with mythical dimensions, a three-headed creature of sorts, a type of sound that has by now become a signature of Hecker’s approach to abstraction. Heavily distorted voices filter through numerous other sounds, resulting in clipped, crackling, leaky speech that gives a surprisingly high-res sensation of what it actually means to be a chimera—a hybrid being, fully functional, yet without a certain place in the order of things. Since you know, with your own body, what having a voice entails, you can literally feel the synthesis taking place, as if your own vocal cords and ears were being visited by unwanted guests.
From 2011 onward, speech-oriented chimeras multiplied in Hecker’s work. In projects such as Chimerization, 2011; Articulaçao, 2012; and A Script for Machine Synthesis, 2015—all collaborations with philosopher Reza Negarestani—public “voices” such as those of Charlotte Rampling, Suely Rolnik, Joan Jonas, Joan La Barbara, and Andreas Huyssen were divested of their normal auditory identity. Recording their speech in anechoic studios that robbed it of reverberation, the normal signature of a specific acoustic location, Hecker interbred the resulting voice files with other sounds—for instance, by sieving a voice’s unique timbre through the specific attack-sustain-and-decay profile of a sound from a very different type of source. The results—wavering, shape-shifting, at once crisp and murky, hitting the ear from many directions at once—give a new sense to Roland Barthes’s concept of the “grain of the voice,” his subterranean vision of the new life of sound. These are synthesized living voices that, corresponding to neither language nor music, complete the step from aesthetic fantasy to the hardworking machine voices of a new and monstrous everyday.
Chimeras are, in other words, embedded in daily life. If Leckey’s 1999 video Fiorucci Made Me Hardcore hit home with so many, it was not simply because of his use of nostalgic footage of British working-class youth shining on the dance floor. It was rather because the clever disconnect between the scenes of dancing and the soundtrack conveyed a visceral sense of bodies shedding their spatial determinants, whether corporeal, cultural, or both. Hecker’s lingering attachment to club culture, evinced by his DJ collaborations with musicians such as Aphex Twin, among others, likewise anchors his work of synthesis and resynthesis in social reality. As audiences entered the Teijin Auditorium at Amsterdam’s Stedelijk Museum to attend A Script for Machine Synthesis—a 2015 sound-voice drama featuring a melting pink ice cube as the main onstage character—each individual received a small rubber perfume disc created especially for the occasion by Frédéric Malle Éditions de Parfums and Carlos Benaim of International Flavors & Fragrances The idiosyncratic synthetic smell spread across the auditorium, the scent of abstractions remembered and abstractions to come. For even as scent, like timbre, may be subject to—and the product of—the most minute forms of scientific analysis and composition, it is, and remains, evanescent, almost impossible to describe. And, just like timbre, it is all around.
Ina Blom is a professor in the Department of Philosophy, Classics, History of Art and Ideas at the University of Oslo, and Wigeland Visiting Professor at the University of Chicago.