Ensemble Metatone
In Ensemble Metatone performances, a server receives and analyses touch data from each iPad in the ensemble and uses machine-learning techniques to classify each player’s touches as one of nine gestures. Over a number of classifications, the server gains a picture of the whole ensemble’s musical direction. Are players converging on one idea? Are they diverging into two parts? Is one player mirroring another? Of particular interest to Charles are “inflection points” in improvised performances, where all players spontaneously move to a new idea. The Metatone server is able to identify such points and reward the performers by unlocking new sounds or tonalities in the iPad apps.
A series of prototype Metatone iPad apps have already been trialled in a number of successful performances including at Electrofringe 2013 in Newcastle, Canberra’s You Are Here festival in 2014 and at the 2013 Percussive Arts Society International Convention (PASIC) in Indianapolis, USA.
In 2014, Charles is hard at work refining the Metatone server applications and developing new iPad-instruments for his musical works. In addition to investigating the musical impact of this Metatone system and developing more accurate and timely feedback in ensemble performances, Charles is exploring the potential of this system in distributed performances and in music education.
Distributed Performance
In Metatone performances, a networked server mediates the players by responding to their musical gestures. In fact, the server and performers do not have to be in the same physical location. A future goal of this research project is to turn the gesture-recognition and performance tracking software into an internet service that could be used for coordinating players from around the world in simultaneous performance. The ability to give instantaneous feedback on the musical direction of the ensemble will be crucial in distributed performances where regular visual and aural cues are limited.
Enabling Creativity
Musical games are often used in classroom situations for teaching improvisation or composition. Students might be asked to follow certain structural rules like playing ideas a particular number of times or imitating others. With Metatone iPad-apps, ensemble performances could be “gamified” so that participants are automatically rewarded for exploring new musical ideas or forms of ensemble performance.
