Gil Weinberg: Jam’aa for Haile

Artist(s):

• Gil Weinberg

Collaborators:

• Scott Driscoll
• Travis Thatcher

Title:

Jam'aa for Haile

Exhibition:

• SIGGRAPH 2006: Electronically Mediated Performances

Medium:

Interactive performance with an improvisational robotic percussionist

Category:

• Performance

Artist Statement:

Computer-supported interactive music systems are hampered by their inanimate nature. They cannot provide players and audiences with the physical and visual cues that are essential for creating expressive musical interactions. Such systems are also limited by electronic reproduction and amplification of sound through speakers, which cannot fully capture the richness of acoustic sound.

Our approach for enhancing and enriching human-computer musical interactions is to utilize an anthropomorphic mechanical apparatus that can convert digital musical instructions into acoustic and physical generation of sound. We believe that musical robots can combine the benefits of computational power, perceptual modeling, and algorithmic music with the richness, visual interactivity, and expression of acoustic playing. Interactive musical robots can bring together real-time analysis and response algorithms that are not humanly possible with rich sound and visual gestures that cannot be reproduced by speakers. This kind of novel human-machine interac­tion can lead to new musical experiences, and new music, which cannot be conceived by traditional means.

Our first effort in this area is Haile, a robotic percussionist designed to demonstrate musicianship. We define robotic musicianship in this context as a combination of embedded musical, perceptual, and in­teraction skills with the capacity to produce rich acoustic responses in a physical and visual manner. Haile listens to live players, analyzes perceptual aspects of their playing in real time, and uses the product of this analysis to play along in a collaborative and improvisatory manner. Haile can therefore serve as a test-bed for novel forms of musical human-machine interaction, bringing perceptual aspects of computer music into the physical world both visually and acousti­cally. “Jam’aa” is a composition for this type of anthropomorphic robotic percussionist and two human players, designed to showcase Haile’s mechanical, perceptual, and musical interaction skills.

Technical Information:

Haile listens to audio input via a microphone installed in each drum. Its low-level perceptual-analysis algorithms address aspects such as note onset, pitch, and amplitude detection. Haile can also detect rhythmic beat, density, accuracy, and a number of high-level percep­tual aspects such as rhythmic stability and similarity. Based on these detected features, Haile responds by utilizing six interaction modes (programmed in Max/MSP). Some of these modes, such as imita­tion, stochastic transformation, and perceptual transformation, are sequential. Others, such as beat detection, simple accompaniment, and perceptual accompaniment, are synchronous.

Haile responds by operating its mechanical arms, adjusting the sound of its hits in two ways. Pitch and timbre variety are achieved by striking the drumhead in different locations, while volume variety is achieved by hitting harder or softer. The robotic arms’ main hard­ware components consist of a linear slider driven by a gear motor and hitting mechanisms that utilize a solenoid and a linear motor. Haile’s hitter can strike at about 15 Hz with approximately five notice­able volume levels. It can be moved from lowest to highest pitch at 3-4Hz.