SoundGuides: Adapting Continuous Auditory Feedback to Users

May 5, 2016 #GMR #Motor Learning #Sonification

SoundGuides is a user adaptable tool for auditory feedback on movement. The system is based on a interactive machine learning approach, where both gestures and sounds are first conjointly designed and conjointly learned by the system. The system can then automatically adapt the auditory feedback to any new user, taking into account the particular way each user performs a given gesture.

SoundGuides is suitable for the design of continuous auditory feedback aimed at guiding users' movements and helping them to perform a specific movement consistently over time. Applications span from movement-based interaction techniques to auditory-guided rehabilitation. We first describe our system and report a study that demonstrates a 'stabilizing effect' of our adaptive auditory feedback method.

SoundGuides has been presented as Late-Breaking Work at CHI'16 [francoise2016soundguides].

Max Example

A Max project is freely available online to demonstrate the mapping-by-demonstration system using Gaussian Mixture Regression:


soundguides screenshot

Reference

  • Jules Françoise, Olivier Chapuis, Sylvain Hanneton, and Frédéric Bevilacqua, “SoundGuides: Adapting Continuous Auditory Feedback to Users,” in Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), San Jose, CA, ACM, 2016, pp. 2829--2836. DOI: 10.1145/2851581.2892420. http://dl.acm.org/authorize?N03766.
    Abstract
    We introduce SoundGuides, a user adaptable tool for auditory feedback on movement. The system is based on a interactive machine learning approach, where both gestures and sounds are first conjointly designed and conjointly learned by the system. The system can then automatically adapt the auditory feedback to any new user, taking into account the particular way each user performs a given gesture. SoundGuides is suitable for the design of continuous auditory feedback aimed at guiding users' movements and helping them to perform a specific movement consistently over time. Applications span from movement-based interaction techniques to auditory-guided rehabilitation. We first describe our system and report a study that demonstrates a 'stabilizing effect' of our adaptive auditory feedback method.
    Download Project Page
    Acceptance Rate: 20%
Featured image