Augmented Conductor

In order to provide better integration of realtime signal processing for live sound generated from acoustic instruments based on the control gestures of a conductor, the Augmented Conductor project sought to place sensors on the conductor's arm(s). This project presented many challenges in both hardware and software including creating chains of sensor nodes, improving on previous sensor designs, capturing and processing the sensor data, and gesture recognition. We used sensors and gesture recognition algorithms from previous Numediart projects.
This project required three parallel research paths: sensors, control data processing, and gesture recognition. The previous sensor nodes were improved by reducing their size while increasing their sensitivity using new gyroscope sensors. The data was processed from raw accelerometer, gyroscope, and magnetometer data into more useful arm angles. The gesture recognition software was improved by adding gesture editing capabilities using both video and sensor values. The integration of all of these research paths allowed the improved sensor data to generate more useful sensor information which could then be used to make the gesture recognition more robust. The entire package was delivered to the composer for composition, rehearsal, and, finally, a performance using the sensor nodes and corresponding algorithms.