scoreLight: laser-based artificial synesthesia instrument
Summary
"scoreLight" is a prototype musical instrument capable of generating sound in real time from the lines of doodles as well as from the contours of three-dimensional objects nearby (hands, dancer's silhouette, architectural details, etc). There is no camera nor projector: a laser spot explores the shape as a pick-up head would search for sound over the surface of a vinyl record - with the significant difference that the groove is generated by the contours of the drawing itself. Sound is produced and modulated according to the curvature of the lines being followed, their angle with respect to the vertical as well as their color and contrast. Sound is also spatialized (see quadrophonic setup below); panning is controlled by the relative position of the tracking spots, their speed and acceleration. "scoreLight" implements gesture, shape and color-to-sound artificial synesthesia; abrupt changes in the direction of the lines produce trigger discrete sounds (percussion, glitches), thus creating a rhythmic base (the length of a closed path determines the overall tempo).
The hardware is very unique: since there is no camera nor projector (with pixellated sensors or light sources), tracking as well as motion can be extremely smooth and fluid. The light beam follows contours in the very same way a blind person uses a white cane to stick to a guidance route on the street. Details of this tracking technique can be found here. When using the system on a table (as in the image on the right), the laser power is less than half a milliwatt - half the power of a not very powerful laser pointer - and does not supposes any hazard. More powerful, multicolored laser sources can be used in order to "augment" (visually and with sound) facades of buildings tens of meters away - and then "read aloud" the city landcape.
It is still too early to decide if this system can be effectively used as a musical instrument (has it enough expressivity? can we find a right balance between control and randomness?). However, it is interesting to note that "scoreLight", in its present form, already unveils an unexpected direction of (artistic?) research: the user does not really knows if he/she is painting or composing music. Indeed, the interrelation and (real-time) feedback between sound and visuals is so strong that one is tempted to coin a new term for the performance since it is not drawing nor is it playing (music), but both things at the same time... drawplaying?
Movies
Collaborative play
References
- Cassinelli A., Manabe D., Perrin S., Zerroug A. and Masatoshi I.:?scoreLight & scoreBots, ACM CHI'12 (Interactivity), May 5-10, 2012, Austin, Texas, USA (2012) [PDF-1MB]
- A. Cassinelli, Y. Kuribara, A. Zerroug, D. Manabe and M. Ishikawa, scoreLight: playing with a human sized laser pickup, International Conference on New Instruments for Musical Expression (NIME2010), 15-18th June 2010 Sydney, Australia, pp:144-149, (2010) [PDF-2.5MB], Slide presentation (without videos) [PDF-11MB]
- A. Cassinelli, Y. Kuribara, D. Manabe and M. Ishikawa, scoreLight, Digital Content Expo 2009 Symposium (25 October 2009, Miraikan - Museum of Emerging Science and Innovation, Tokyo): Slide presentation [PPT].
- A. Cassinelli, Y. Kuribara, D. Manabe and M. Ishikawa: scoreLight: a laser-based synesthetic experience, additional documentation for SIGGRAPH ASIA 2009 (Art Gallery), [PDF-1.7MB].
- Check a more complete page here: http://ishikawa-vision.org/perception/scoreLight/scorelight.html
