Artificial Intelligence in Composition
Recently, I played a series of symphonic movements for a class. Some were by Mozart, and others by other composers. With a little practice and guidance, the class picked up a rough impression of Mozart’s style, as distinct from the other works. The last piece I played was by David Cope’s software Experiments in Musical Intelligence (better known as EMI or Emmy). Emmy was designed to emulate other composers’ styles as closely as possible, and I wanted to test its effect on a class that wasn’t aware any of the music they would be hearing was written by software.
Of course, my real purpose was to test their reactions to algorithmic composition in general. One student, who’s preparing for her final school exams, gave a comment that’s been fairly exemplary of those I’ve heard when I bring up the topic: “You want to know that there’s a person writing the music. Otherwise how can it be special?”
Cope mothballed Emmy in 2003, and has channelled much of his subsequent work into another algorithmic composition project, Emily Howell, which uses outputs from Emmy and Cope’s training with an association network to generate music in its own style. It was only when I played Emily Howell’s music for the class that that same student was taken aback. She knew the piece. It’s in her study playlist.
The physicist Richard Feynman once said of the computer that it’s “only because it is able to do things so fast that we do not notice that it is doing things very stupidly.” Emmy, like all software, is built to solve problems. Feed it a few pieces of music, and it will analyze the music for melodic and harmonic progressions, analyze the structure, key changes, tempo changes, then take apart the work you have fed it and rebuild it into something new, like the world’s most complex jigsaw puzzle.
The result is surprisingly effective. In his book, Virtual Music, Cope quotes a student who, along with about half his class, attributed a Chopin Mazurka to Emmy, and Emmy’s Mazurka to Chopin. The student wrote of a “collective gasp and an aftermath of…delighted horror.” What a perfect phrase to describe the experience.
The major criticism that Emmy has tended to receive has been that it often quotes entire passages from the composer it’s imitating. While Cope defends this by saying that composers often quote their own work anyway, the extent or fidelity of the quotes tends not to be as extreme as in Emmy’s work. In sight reading the third movement of Emmy’s Beethoven Sonata, I was struck by a passage, several bars long and taken, almost without alteration, though in a new key, from the second movement of Beethoven’s Sonata No. 30. Playing it felt like seeing a familiar location in a science fiction movie—fitting for where it was, but too familiar to ignore. Likewise, when I played the Chopin mazurka for a friend, she instantly recognized the opening as having come from one of Chopin’s Waltzes, “which is really odd,” she said, “because Chopin didn’t do that.”
The tendency to over-quote composers it’s imitating is surely a short-term one. Because Emmy is an early attempt at a musical AI, there are still ways that it can be improved. But far more interesting than the quotes from works that Emmy has been fed are the references to works that it hasn’t. In the same Beethoven Sonata, Cope points out, Emmy seems to reference Mozart’s C minor “Fantasy”—which Beethoven greatly admired—and a discarded theme from the “Waldstein” Sonata.
Perhaps it shouldn’t be all that surprising that this first generation of imitative algorithm seems to be referencing works from outside its own inputs. The composers fed to Emmy do have distinctive styles and tastes, and so some melodies will come to them naturally. Moreover, humans are pattern-matchers; we see faces where faces don’t exist, and at least some of Emmy’s accidental references can be attributed to this false pattern-matching (as, surely, can some of the quotes found in human composers’ work). But these accidents of fortune can be as much a part of a composer’s style as deliberate choices can, provided they’re used well.
Perhaps future generations of software, given more inputs and more scope for what to do with the outputs, will not only make these references accidentally, but then go on to use them effectively, as human composers do. For now, it’s up to the human composers to recognize these links. People are an extremely important part of the work these algorithms do: they must create the databases, and must judge the results. Cope argues that this work can only be done by composers.
While a computer that can write like Mozart, or one that can paint like Rembrandt, is remarkable as a show of programming skill, it’s of limited practical use. No matter how good the Mozart or the Bach or the Beethoven it makes, we’ve got an enormous amount of Mozart and Bach and Beethoven already. And maybe the true test of these types of imitative algorithms will be in the completion of partially-finished works (such as Emmy has done with Beethoven’s Symphony No. 10). In some ways, algorithmically-generated completions of these works may be the truest completions of them we’ll ever have. A computer’s input that consists solely of a single composer’s work can only be influenced by that composer. Any living composer trying to write in the style of another has to discard thousands of other hours of music he or she has heard—and that may not be possible.
If there’s a threat to come from software like Emmy, it will probably at first be an economic rather than an artistic one. It’s likely to hit the entertainment industry first—movies and TV—since the only concern there is with effective music. What director would say no to a soundtrack by John Williams? Well, maybe the director who’s been offered a soundtrack by Brahms. Cope argues that the composer will still be necessary to the process. “Frankly,” he told me, “I know a number of film composers that aren’t the slightest bit concerned given the lack of musicianship on the part of producers and directors.” He considers Emmy a tool—a “shovel” for composers to help them with their work, rather than a technology that will make human composers obsolete.
The job of tools is to help people work faster. The side-effect of this is that you need fewer people to do the same amount of work. For now, according to Emmy’s creator, the lighter compositional load may be balanced out by the extra work at the beginning and end of the process, the choosing of the works to act as a database for the software, and the choosing and tweaking of the best output. In this way, the composer becomes, in part, a curator and director as well as an organizer of sound. Some observers, like Bayan Northcott in an Independent article from almost 20 years ago, call this a cheat. The computer’s only keeping what a human says is good. But if it’s a cheat, it’s the same cheat Leopold Mozart used to sculpt the image of his son as a boy genius.
Emmy’s simulations of composers, while convincing, may not be perfect yet. When I made the point to Cope that we can also teach students to compose like Mozart or Bach or whoever, he said, “I think I can do better, but why would I try? Emmy’s output is plenty good at it and I’d rather create my own music in my own style as a composer.” But in this way, Emmy functions as a proof-of-concept. Already, from atomized Mozart, Emmy can create virtual Mozart. What’s to happen with the next generation of algorithmic music generator? It will surely take the same processes to the next analytical level.
The next generation may be close. On June 1, fresh off their astonishing success creating a computer that could play Go at world-class level, Google’s Brain team announced their next project: to have a computer create a new piece of music, including—and this is the major aspect not really touched by Cope’s work—the structure. They’re calling it Magenta. In their announcement blog post, Google mention structure and surprise specifically, calling out Cope and Emmy by name: “So much machine-generated music and art is good in small chunks, but lacks any sort of long-term narrative arc. ...Alternately, some machine generated content does have long-term structure, but that structure is provided TO rather than learned BY the algorithm. This is the case, for example, in David Cope’s very interesting Experiments in Musical Intelligence.”
Go was long considered a far harder game to teach a computer than chess. Deep Blue, the IBM program that beat Garry Kasparov, played chess essentially in the same way a human does: calculating as many moves ahead as it can in every possible scenario, and choosing the optimal one. Google’s AlphaGo was initially taught to play this way, but the number of possible moves in Go is far greater than in chess, and AlphaGo eventually had to learn the game in a more human way: playing many matches, and learning from experience. When it beat the human player Lee Sedol, one of the most remarkable things about its play style was that it chose moves that seemed odd until viewed in retrospect, and according to Ko Ju-yeon, a professional player who attended the match, it “seems to have totally original moves it creates itself.” Its strategy is both original and utterly effective.
If Google or someone else can develop a bot with the same level of originality in music composition as AlphaGo has presented in Go, that’s when the real philosophical and existential questions begin. Every opinion we’ve had about art in history has been built on the (obvious) assumption that it’s been made by people. Within the next few years, in a very real way, that may no longer be true. Machine learning is weird science, and its results are hard to predict. Unlike Go, there is no victory in composition. There’s no world championship to win. Instead, Magenta has to make something that people will enjoy, and that’s a coherent piece of work.
It will be a hard challenge for Google, but I wouldn’t bet against them.
The Go community has responded to AlphaGo as an opportunity to improve its own games. Computers can show paths that people hadn’t seen before, and people can learn from these paths. In the same way, composers will be able both to compete and to collaborate with computers, and in doing so discover new ways of writing and of building music; new ways to express themselves in a new world.
If there’s hope for human composers and for human art in the long term, it’s this: art is more than art. There is a human connection born from knowing the circumstances of composition—Mahler hearing a firefighter’s funeral from his New York hotel while writing his Symphony No. 10; Sibelius watching a flock of geese fly overhead as he wrote his Symphony No. 5—can make the work more significant. When I asked Cope about these connections, he said that it “will be different in the case of computers than it will for composers”—computers won’t evoke feelings in the same way, but they can elicit feelings nonetheless. Listening to one of Emily Howell’s fugues, I found myself surprised at how I was moved by it. Still: the connection between creator and audience is fundamental to how art works.
Well, until AI can perfectly duplicate human consciousness, and then we’ll have much bigger philosophical questions to worry about than whether its music is any good. ¶