What are the Adaptive Use Musical Instruments (AUMI)?

Adaptive Use Musical Instrument (AUMI) software interface and iOS App are new musical instruments that enable people who have very limited controlled (voluntary) movement to independently engage in music making.

AUMI enables the user to play sounds and musical phrases through movement and gestures. This is an entry to improvisation that enables exploration of sounds ranging from pitches to noises rather than learning set pieces. This open approach to music enables anyone to explore and express a range of affects, both by themselves and in response to, or in conversation with, others. While AUMI can be used by anyone, the focus has been on working with children who have profound physical disabilities.

In taking these participants as its starting point, the AUMI project attempts to make musical improvisation and collaboration accessible to the widest possible range of individuals. This approach also opens up the possibility of learning more about the relations between ability, the body, creativity and improvisation, from within a cultural context that does not always acknowledge or accept people with disabilities.

The AUMI project continues to be revised and improved with input from the technologists, students, therapists and feedback from registered users.  An on-site training program is now available. The latest initiative is the development of an AUMI iPad App.


Led by musician, composer, and humanitarian Pauline Oliveros, the Adaptive Use Musical Instrument (AUMI) project brings together the expertise of technicians at the Rensselaer Polytechnic Institute (RPI) and the community education initiatives of the Deep Listening Institute.

In collaboration with musician and occupational therapist Leaf Miller, the AUMI software interface was first used in drum workshops with children with disabilities at Abilities First School located in Poughkeepsie, New York in 2007. Since these initial workshops, the software interface has been made available as a free internet download and is now in use by musicians and therapists both nationally and internationally.

AUMI was adopted by the Gender and the Body research group of an international research project on Improvisation, Community, and Social Practice (ICASP). ICASP is centered at the University of Guelph, Ontario, Canada and directed by Ajay Heble. ICASP researchers are exploring improvisation as a model for social change in a variety of teams drawn from 33 researchers.

In 2013 the first six years of ICASP concluded with AUMI named as a top rated project and transformed to a new project: International Institute for Critical Studies in Improvisation (IICSI). Gender and the Body research group continues with research on AUMI and its value to students with special needs as part of IICSI through 2020.




The mission of Center for Deep Listening at Rensselaer includes the development of technology for artists. AUMI was created to support artists with disabilities. The starting point was to work with those potential artists with the least possibility to participate. This point of view has made artistic expression available to a wide range of people who would otherwise have no access. AUMI is an inclusive interface that all can use. More research is needed in order to design even more effective features to support an inclusive range of music making. CCC provides a training program in the use of AUMI. Feedback from those who engage in the training programs help to inform the next developments of AUMI.

Though AUMI is very intuitive for users to learn on their own there are many aspects of ergonomics, environments as well as artistic considerations that support free improvisation as the preferred way to use AUMI.

Technology and Training Research

 CCC-AUMI employs a Participatory Action Research approach, which provides direct feedback about the effectiveness of the musical instrument in a variety of creative, educational and therapeutic settings. This user feedback from all learners and educators helps push the interface forward to wider ranges of expressiveness and use.

We are continuing our research on smart hands free controllers for interfaces. We want the technology to adapt to the user. Currently we are updating all software to run on new systems and updating all versions of AUMI with new features. Our trainer is continually updating the manual and creating a curriculum for therapists, parents, and aides to use in facilitating improvisation with AUMI.

AUMI-RPI CDL/CCC is a Member of the AUMI Consortium, an international research group dedicated to exploring, sustaining, developing, and sharing the Adaptive Use Musical Instrument. Each AUMI Consortium Member institution has a particular area of focus. Ours is Adaptive Use software and hardware design and development to enable people with disabilities to improvise and training for educators, therapists and parents to use AUMI with their students, 
clients and children.

AUMI Consortium Members:

  • AUMI – Rensselaer Polytechnic Institute
  • AUMI -Carleton University
  • AUMI – Kansas University InterArts 
  • AUMI – Lakehead University
  • AUMI – McGill University
  • AUMI – Memorial University

Additional Researchers include:

  • Center for Deep Listening at Rensselaer, Troy, New York
  • Staff and students of Abilities First, Inc., Poughkeepsie, New York
  • Scholars and students of ICASP from the University of Guelph, Guelph, Ontario
  • Faculty and students of Easter Seals, St. John’s, Newfoundland
  • Faculty and Students of McKay School, Montreal, Quebec
  • Faculty and Students of Centro Camino, Argentina

AUMI Programmers include:

  • Zane Van Duzen
  • Zevin Polzin
  • Doug Van Nort
  • Jaclyn Heyen
  • Ian Hattwick
  • Henry Lowengard



AUMI 3.0 and 2.0 for PC and Mac is available here: AUMI download registration

AUMI 1.1 iOS for iPad and iPhone is available at the Apple Store.

  • Note on the app:  AUMI 1.1 is happy in IOS8, runs on iPhone 4,4s,5,5s,6 and 6+ as well as iPads that have cameras in them. The Adaptive Use Musical Instruments (AUMI) software interface enables people who have very limited controlled (voluntary) movement to independently engage in music making.

Developed by Henry Lowengard, this iOS version of AUMI runs on devices with front facing cameras, and can run on iPhones (long and short), iPod touches and iPads. A large assortment of sounds can be triggered with configurable amounts of sensitivity. AUMI also can be used as a MIDI controller to access other iOS based synthesizers or external equipment(with the iPad’s Camera Connection kit or via Core MIDI over WiFi.

aumiapp_screenshot1 aumiapp_screenshot3 aumiapp_screenshot4 aumiapp_screenshot2

click on image to enlarge

AUMI captures a record of the movement in the session, which can be exported and turned into reports in a clinical setting.

AUMI iOS User Manual

Read reviews on: Apps4 iDevices / Zaclaurent.com



AUMI Manuals Available for Download

AUMI 3.0 Manual: AUMI 3 Manual

AUMI 2.0 Manual: AUMI 2.0 How To Guide

AUMI Manual – Greek: AUMI_gr


AUMI 3.0 Video Tutorials

Introduction to AUMI Beta 3.0

Introduction to AUMI Beta 3.0 from Deep Listening Institute on Vimeo.

AUMI Beta 3 Keyboard Tutorial

AUMI Beta 3 Keyboard Tutorial from Deep Listening Institute on Vimeo.

AUMI Beta 3 Quarter Screen Tutorial

AUMI Beta 3.0 Quarter Screen Tutorial from Deep Listening Institute on Vimeo.

AUMI Beta 3 Relative Movement Tutorial

AUMI Beta 3.0 Relative Movement Tutorial from Deep Listening Institute on Vimeo.

AUMI Split Screen Percussion Tutorial

AUMI Beta 3.0 Split Screen Percussion Tutorial from Deep Listening Institute on Vimeo.

AUMI 2.0 Tutorial