| Form, Substance, Correspondence:
Intersensory Composition in Digital Media Paul Hertz, Northwestern University paul-hertz@northwestern.edu |
<back
to
program |
||
|
Wagner's "total art work" (Gesamtkunstwerk) and the correspondence
of
sound, color and scent proposed in Charles Baudelaire's poem
"Correspondances" influenced a whole generation of artists in their
time. Their influence is still felt today, when digital media offer
us new possibilities for creating multisensory art. With the computer
we can compose and perform multisensory events with a degree of
precision beyond the threshold of human perception. This capacity is
not just of interest to artists: immersive interfaces and
multisensory representations of data have become an important tool
for scientific research and the exploration of large data sets. In
their separate ways, both scientists and artists attempt to make
sense of the world. Arguably, their methods if not their styles
converge in the propagation of digitally-mediated multisensory
experiences.
This paper contends that a specific area of these experiences, which
we shall call digital intermedia, can prove particularly fruitful.
"Intermedia" is a term coined by the Fluxus artist and theorist Dick
Higgins. In intermedia, the compositional process works across the
boundaries between media or even fuses media. While it is sometimes
called "synesthetic art," intermedia does not seek to imitate the
physiological phenomenon of synesthesia. It extends the creation of
form across sensory modalities without necessarily promoting a tight
coupling of multisensory events. At the same time, intermedia implies
something more specific than multisensory opera. It is not so much
a
"total art work" as a hybrid art work.
Now that digital media enable us to generate multisensory experiences with all sorts of structural couplings between modalities, questions old and new arise. Can a synthetic experience of sensory coupling stimulate the heightened mental states sometimes associated with neurological synesthesia? Can cross-sensory structures actually be perceived, or are they just a compositional device? Whether perceived as forms or not, will cross-sensory composition prove to be memorable in the same ways that paintings or music are? We would like to suggest that the answer to these questions is a
qualified "yes." Intermedia can be considered a form of immersion.
Not unlike the 3D immersion of virtual reality, it involves a
threshold experience, a point at which the senses become accommodated
to artifice. While virtual reality involves problems of latency for
which we have laboratory measurements, immersion in VR also has
elements in common with immersion in a musical work, or with the
willing suspension of disbelief in reading a novel. All of them
represent moments when our cognitive system adjusts to and becomes
absorbed in a particular form of representation. The scientist who
repeatedly explores a data set with visual, sonic and possibly haptic
mappings is not unlike the person who listens repeatedly to a
symphony, hearing new things each time. Only the emotional responses
are distinct. Whether we are looking into the inner processes of the
physical world or the inner processes of the imagination,
digitally-mediated multisensory experiences will open us to new
insights. |
|||
|
|
|||