Designing Systems for Human Interaction, Not Human-Computer Interaction

by Camille Utterback

I am a 32 year old artist, in a booming metropolis, at the dawn of the 21st century. I am armed with a couple of computers, a pile of camera equipment, some programming skills, a good exhibit track record, my own company, and an amazing network of friends and colleagues. I want to change how people think about their relationship to technology. I want to bring our physicality, our bodies (a large part of what makes us human) back into the equation. I am passionate about creating experiences that show people that their interactions with computers do not have to be frustrating, deadening, and potentially debilitating. Instead, we can imagine and create a world where this interaction is seamless, intuitive, playful and inspiring. By using video cameras to create physical-digital systems that engage people's bodies instead of just their fingers and eyes, I hope to refocus attention on the embodied self in an increasingly mediated culture. Additionally, my video-based interfaces, by allowing many users simultaneously, create social spaces focusing on human interaction, not human-computer interaction.

I have exhibited my interactive video installations at art galleries and festivals around the world, and in science museums, children's museums, and corporate and retail environments. My pieces have even found their way into private collectors' living rooms. When I see the look on people's faces as they watch projected imagery magically react to the image of their body--as they giggle and start doing odd contortions with their body and begin playing with people around them--I know my vision is possible.

When historians look back at these early physical-digital experiments that my many colleagues and I are creating now, perhaps these pieces will look like early daguerreotypes--lacking polish and definition--but I think we've begun down a path that will be remembered as the beginning of an evolution in design. In Donald Norman's book, The Invisible Computer, he points out that we no longer buy "motors" (as was once the case). We buy "blenders" or "hair dryers". When motors became cheap enough, they disappeared into consumer objects designed for specific purposes, and were no longer purchased as separate objects. Artists and designers like myself are beginning to put an end to the "computer". Physical-digital objects will evolve and proliferate, taking many forms from consumer objects, to custom designed exhibit design, to art work. Physical-digital interfaces will not be novel, they will just be used. The appeal of my work lies not in the fact that it uses technology in a new way, so much as it allows the technology to disappear, and lets people enjoy a physically-based, human-centered experience.

So what do I make? As mentioned, most of my interactive installations to date use a video camera as an input device. Video data provides much richer information about the world than a mouse and keyboard input device do. I write custom software to process the incoming camera signal to glean information about users' positions, motions, or gestures in the installation space. Imagery is then generated and projected in response to the camera input. The effect is that the images on the screen appear to magically and transparently respond to people in the space. A secondary, but no less important effect, is that using a video camera as an input allows for many people to interact with my systems at once. This helps my installations to become social spaces where people interact with each other as much as with the system I've developed.

I have developed a number of installations (www.camilleutterback.com, www.creativenerve.com) that respond to users in different ways, but for this article I will concentrate on discussing Text Rain, created with Romy Achituv in 1999. 'Text Rain' is the oldest, and perhaps most well known of my video tracking based works.

In the Text Rain installation, visitors see a live black and white video projection of themselves. Colored letters drift down the screen like rain or snow and can be caught and lifted by people's heads, limbs, or any other object in the video image. As one set of colored letters fall, they fade and are replaced by new letters from above. The lines that fall are form a poem about bodies and language. If participants catch enough letters they can read words and phrases from the poem.

The video-tracking interface design ofText Rain feels "transparent" for a number of reasons. Text Rain responds immediately to a person's presence, whether or not that person intends to interact with the piece. There is no donning of complicated headgear, and no queuing up for your turn. Because the boundary between being "in" or "out" of this virtual space is only a matter of attention, users can talk and engage freely with other people in the installation space, while simultaneously playing with the letters in the virtual space. The boundary between the real space and the virtual space is "thin" because it is easy for users to be present in either the real or virtual space, to seamlessly shift between the two, or to feel present in both simultaneously. The interaction with this system also takes advantage of the fact that people already know how to manipulate their bodies--particularly in front of something that resembles a mirror. Participants are not asked to learn a new metaphor to operate the interface. If we want to catch something in the real world, we hold out our arms--which is exactly how the interaction works in Text Rain. A familiar physical activity is translated directly into the system.

There are many systems other than Text Rain that use video as an input. Myron Krueger is considered by some as the father of this field. Scott Snibbe's Boundary Functions is a more recent, but beautiful example.) Part of Text Rain's success, however, also depends on the specific design of the system, not just the video-tracking aspect of it. While the reaction of the letters to users is immediate and easily understood, the piece also sets up an open-ended dynamic that allows for a wide variety of personal discovery and play. There is no right or wrong way to interact with this piece. As creators of this work, Romy and myself have been continually amazed watching people engage with it. In order to catch more letters, people have opened umbrellas, stretched scarves between them, and enlisted a series of strangers to hold hands across the screen.
The social dynamic set up by the system is also very open ended. People communicate with each other directly in the real space by turning to talk with each other, or via the virtual space, by connecting gazes through their on-screen images. People cooperate to try to catch more letters, or joke around, stealing letters from each other. Far from creating a virtual world in which users are "lost," Text Rain, through the virtual space it creates, activates the real space in which it exists. The on-screen space created by the Text Rain installation encourages participants to move and gesture with their body in the real space. The virtual space also encourages social interaction within the real space of the gallery.

I hope that my contribution to the world will be to notch open the door a little bit, to create a chink of light in the wall of prevailing design--the unmalleable gray and putty (now happy iMac-colored) boxes that confront us with their glowing screens and carpel tunnel syndrome. If these machines will underpin our future work and play, how can we design them to respond to the full range of human embodied existence?

>> back to the physical computing roundup

Camille Utterback has exhibited her award-winning interactive video installations at festivals and galleries internationally. Notable accolades include a Rockefeller Foundation New Media Fellowship (2002-2003) and inclusion in the 'TR100 top 100 innovators of the year under 35' by MIT's Technology Review (2002). Utterback holds a BA in Art from Williams College, and a Masters degree from The Interactive Telecommunications Program at NYU's Tisch School of the Arts. In addition to creating her own artwork, Utterback develops installations for commercial and museum settings via her company Creative Nerve, and teaches at the Parsons School of Design.