Computers `look' and `listen' as they play

When it comes to performing on stage, must computers be deaf, dumb, and blind? Researchers at the Experimental Music Studio of Massachusetts Institute of Technology don't think so. They're turning computers into ``synthetic performers'' capable of acting as informed and musically responsive members of an ensemble.

In a Handel flute sonata, for instance, the computer knows the part played by the flutist, and it knows the harpsichord part that it must play. According to the studio's founder, Barry Vercoe, ``It must figure out how the two parts fit together.''

Through optical sensors (small devices attached to the keys of the flute), the computer watches the fingers of the flutist, and through pitch-tracking, it listens to the music in the same way a harpsichord accompanist would. It then plays along with the flutist in a musically intuitive way.

For example, when the flute player slows down the tempo, the computer senses this and adjusts its speed to correspond. When the flutist speeds up, the computer follows suit, because it knows the score.

What if the instrumentalist pauses slightly before one note in order to add a musical nuance to a phrase?

As it turns out, the computer fumbles momentarily, but eventually regains its place in the accompaniment. When the musician does it again in the same place -- on a second run-through -- the computer anticipates the gesture and slows down to accommodate the musician's interpretation. The synthetic performer, then, ``learns'' from rehearsing.

Although certain kinks need to be worked out, this form of computer music brings a measure of relief to the tape-plus-instrument syndrome, where musicians performing live must keep pace with a recorded computer part. Even those musicians who work regularly with loudspeakers say they can come off sounding neutral and non-human.

The synthetic performer is helping to overcome the invariant nature of computers, says Mr. Vercoe. He has wanted to break out of the medium of the computer as a ``dumb processor,'' he says.

``The computer music research people have an obligation to make technology musically responsive, else the commercial market will wipe out all the artistic residue we have.''

``We can't presume [the computer] has a fix on who you are,'' he says. But the synthetic performer is an attempt to account for the give-and-take between musicians -- the subtle nuances and variations that make every performance unique.