Robo-music gives musicians the jitters
The Venice (Fla.) Little Theater has a tiny orchestra pit, with room for only a handful of players, and a modest budget. So when it mounts a big musical like "Beauty and the Beast," it brings in an electronic ringer.
A laptop computer, loaded with a program called OrchEXTRA, serves as a "virtual orchestra," from strings to woodwinds, drums to horns, giving the music such a rich sound that audience members may wonder how a full Broadway orchestra fits into the tiny pit.
"As far as sound quality, these things are great," says Dorian Boyd, the sound designer/technician for Little Theater, referring to OrchEXTRA. Virtual orchestras are much better than the early systems of just a few years ago, he says, which could sound like "video game music." Plus, "They're so reliable, you don't really need much of a backup plan," says Mr. Boyd, who works for the nation's fifth-largest community (amateur) theater. It has a 432-seat main stage and 90-seat second stage. "My backup plan is a piano in front of the musical director."
Virtual orchestras – computer programs that can vary dynamics and tempo and follow the singers on stage and the music director's baton – are changing the music world. They are a sore point with Broadway musicians, who fear being replaced by machines and went on strike in 2003 to protest shrinking pit orchestras. But the systems are being used widely with traveling professional Broadway shows and in Cameron Mackintosh's revival of "Les Misérables" in London's West End. Several Cirque du Soleil extravaganzas in Las Vegas and elsewhere use the technology, as do thousands of performances by high school and other amateur groups.
Music students benefit, too. Rarely, if ever, do they have a chance to solo on their instrument accompanied by a full orchestra that will follow their lead. A virtual orchestra created by an Indiana University professor can do just that. It's a great learning experience, says inventor Christopher Raphael, and a great training tool. "You can prepare for performance much more effectively when you hear all of the other parts," Mr. Raphael says. He's an associate professor of informatics at Indiana and former professional oboe player.
His Music Plus One system, under development for 13 years, begins with a recording of an orchestra playing the piece, minus the solo. Computer programming allows the orchestral accompaniment to "listen" to the soloist and follow. The program also uses predictive programming, based on the player's previous playing style and past rehearsals, to anticipate what to do next. Time-warping technology permits the virtual orchestra to slow down or speed up without changing the pitch of the notes being played.
Virtual orchestras have yet to pass the musical version of the "Turing test" – Alan Turing's 70-year-old test for how to tell when a computer's artificial intelligence has become indistinguishable from that of human intelligence. Last August, Britain's Guardian newspaper asked Paul Hughes, general manager of the BBC Symphonic Orchestra, to compare a piece played by a virtual orchestra (stitched together from samples of the recorded sounds of instruments) with the same piece heard on a recording of a human orchestra. Mr. Hughes easily spotted the ersatz orchestra, created by the Vienna Symphonic Library in Austria. Although the playing was "beautiful," he said, it was also "bland." "The computer version was almost too perfect," he said.
Raphael and others working on virtual orchestras say their intent is not to replace human musicians and the joy of making music – or to equal a human ensemble.
"I don't think that [a virtual orchestra] will ever sound exactly the same or as good as a traditional acoustic orchestra," assuming it's performing the traditional European repertoire, says Jeff Lazarus, chief executive officer of Realtime Music Solutions in New York City. His company supplies the technology, called Sinfonia, which underlies OrchEXTRA and several other virtual orchestra products.
A Sinfonia performance of a musical score begins by drawing on Realtime's huge database of sounds of instruments playing notes at various dynamic levels and articulations. In a "painstaking process," human musicians help sculpt the individual lines played by each instrument. For a professional production, the company will meet with the composer or music director ("more accent on beat 3, more legato here") to further refine the sound, as well as seek input from the human musicians that will be playing with the Sinfonia. Even the acoustics of a particular theater can be taken into account.
You don't just "push 'play,' " Realtime's Mr. Lazarus says. A trained musician must control the Sinfonia. "They're reading the score, and they're absolutely a musician playing in the band" who must rehearse their part just as other musicians do.
Realtime came under fire from union musicians on Broadway in 2003 when producers threatened to use virtual orchestra machines to replace musicians during a four-day strike. The technology has not been used there since, even though other high-tech aids – music synthesizers, computerized lighting – are employed, as Realtime points out.
"We don't want to see the virtual orchestra brought in ... not just [in] theater but opera or ballet, church services, or anything [else] because we believe it's a machine and not a musical instrument," says Vicky Smolik, a musician based in St. Louis who is president of the national Theater Musicians Association.
Concern about virtual orchestras, she says, is "a big thing" with her union members, "from New York to Los Angeles ... and everywhere in between." Virtual technology could "replace the whole art form, eventually, if it decided to take over," she says. "And nobody wants to see that because that's why it's called 'live theater.' "
Mr. Lazarus says he's come to loathe the term "virtual orchestra" because it has been "poisoned." "The union has done a very good job of demonizing 'virtual orchestra' as a concept and making it seem like it's the devil ... public enemy No. 1 for musicians."
Realtime doesn't advocate replacing orchestras with Sinfonia, he says. What it can do is allow a small orchestra to sound like a bigger one. The technology works best when live musicians take the lead parts and the computer fills in the background, he says. "You're going to want to get a good [human] fiddle player" to play the key opening solo for a production of "Fiddler on the Roof," he says. But Sinfonia might fill out the background string section for a lusher sound.
Charges that virtual orchestras are taking jobs from human musicians are "totally overblown," Lazarus says. Producers will hire as many musicians as they can afford or can fit into the pit (which they've shrunk to put in more seats) because live musicians offer the best-quality sound, he says. "Ultimately, people are making choices to use Sinfonia for artistic reasons," not financial ones, he says. "They recognize that it's better than the other compromises they're exploring.... Better than just cutting down to a 10-piece orchestra."
When "Les Misérables" moved from London's Palace Theatre to the Queens Theatre in 2004, the new orchestra pit had room for only 11 musicians, not 20-plus, Lazarus says. The show might have closed rather than sound radically different with a tiny orchestra. Mr. Mackintosh decided to augment the smaller group with Sinfonia, despite union protests. "I like to think I've created 11 musician jobs," Lazarus says, as well as saved the jobs of everyone else in the production.
Raphael, a former member of the musicians' union, says he's "totally opposed to the musicians' union position" against the virtual enhancement of orchestras. "Ultimately, what they do ends up giving the world less music, not more music," he says.
He hopes his Music Plus One eventually will be used more widely, not only as a practice aid but also to make new music and open up possibilities for composers. For a virtual orchestra, no notes are too fast to play, no rhythms too hard to understand, he says. "So you can create pieces here that ... would be extremely hard to do with human musicians," he says.
Today's classical pianists already have to compete with historic recordings by the great masters of the keyboard from the first half of the 20th century.
The moderns have had an edge, so far, in the technological sophistication of their recordings. But now that edge may be gone, with new recordings "performed" by legends such as Sergei Rachmaninoff, Arthur Rubinstein, or Vladimir Horowitz.
First to be resurrected is Canadian phenomenon Glenn Gould (1932-1982), whose 1955 recording of J.S. Bach's "Goldberg" Variations is prized by his legion of fans. On Sept. 25 at the Glenn Gould Studio in Toronto, the late pianist once again "performed" the Goldberg. Or rather, a computer-controlled Yamaha Disklavier Pro piano programmed with all of Gould's attacks, keystrokes, pedal work, and other idiosyncrasies captured from the 51-year-old original recording did the performing. A digital recording in surround sound of the "new" performance is expected to be released by Sony Classical in March 2007.
"It's a chance to sit in the room, essentially, as Gould plays, as opposed to hearing it from a mono[phonic] source," says John Q. Walker, president of Zenph Studios in Raleigh, N.C., which developed technology to extract the essence of a pianist's playing style from old recordings.
The new version, Mr. Walker says, is "more immersive. It's more real." He likens it to the difference between HDTV and regular TV.
To capture Gould's style, Zenph analyzed 10 things about every note. What was the microsecond when it was struck? What was its duration? How hard was the note struck, and how was it released?
Then there's "All the stuff to do with the pedals," Walker says. The original recording conditions had to be factored in, too, including where the original microphones were placed, how the sound reverberated off the walls, and how much of the full dynamic range of sound the original recording captured.
At some points, skilled musicians had to help out. "There are a lot of things we don't know how to do automatically, so we have humans guide the process," Walker says.
Zenph has a contract with Sony Classical to produce 18 CDs using its technology. Half will be works by classical pianists, half by jazz pianists. The next artist up for "re-recording" is jazz great Art Tatum.
Eventually, Zenph may reach back as far as a 1903 recording by French pianist Raoul Pugno. That would be "quite doable." Walker says. But recordings made before then, such as one of Johannes Brahms himself playing the piano, aren't good candidates. They're "simply too primitive," Walker says.
