AIM (Artificial Intelligence for Musicians) is a Purdue music technology research group, whose aim is to create reactive, human-like systems which support musicians during their practice sessions and performances.
Some of AIM's projects are supported by a National Science Foundation grant.
We are looking for motivated graduate or undergraduate students to join our team. Click here for info.
Evaluator is an app that aims to help musicians practice more effectively. It utilizes computer vision and YOLO localization techniques to help musicians track, analyze, and improve their posture. It also uses spectrogram analysis and multi-modal transformers to help the musicians identify their mistakes in music and correct them.
Drawing by Cecilia Ines Sanchez.
Companion is an app that not only plays along with a human player during a chamber music piece, but actively responds to their playing habits and voice commands like a real human would. The project involves machine learning and filtering/DSP algorithms to analyze and edit sound quickly and accurately and utilizes small NLP language models for voice command implementation.
Drawing by Cecilia Ines Sanchez.
Mus2Vid is a real-time art project that uses diffusion models to generate video depictions in response to classical music. It uses recurrent and transformer networks to analyze input audio and estimate its emotion and genre qualities, which are converted into text and fed to a text-to-image diffusion model to generate images.
As the name suggests, Robot Cello is a project about using reinforcement learning to teach a robot arm to play cello. The project is currently in its survey phase but is currently investigating using motion capture technology to get training data for an RL model.
We partner with the Purdue Envision Center to collect motion data for our robot arm to train on. On the left is a video of Prof. Yun playing cello while wearing a motion-capture rig.
Our projects often span most or all of these areas, as they are all important to making effective, human-like music technology.
How can Companion utilize machine learning and filtering to resynthesize string instrument articulations on-the-fly?
What are the most effective methods for Companion, Evaluator, and Mus2Vid to follow a musician's playing in reference to a score, and play along to match?
How can Mus2Vid analyze emotion of classical music in real-time and utilize it to generate real-time video accompaniments?
What musical features extracted from various media, such as tempo, key, genre, notes, are useful to music performance technology, and how can we extract such features?
How can our apps be designed in ways that are human-like and natural for humans to interact with?
How can we ensure that our users actually utilize and enjoy the apps we develop?
Our team comes from a wide variety of backgrounds, including the Purdue Colleges of Engineering, Science, Liberal Arts, Management, and more. We have a mix of Purdue professors, graduate students, and undergraduate students leading our projects and research efforts.
Learn moreHere are some of the ways AIM ensures that it is integrated with both Purdue's local community and the global community.
AIM has an associated Vertically Integrated Project, which enables research experiences for Purdue undergraduates.
Music technology is a field with very multidisciplinary problems and solutions. As such, we draw from a wide variety of departments for our talent, including Music, Electrical and Computer Engineering, Computer Science, Technology, Art, Design, and Management.
AIM actively runs user studies on the tools that are developed to ensure that users can figure out how to use them and enjoy them.
AIM members have given speeches, presentations, and concerts about and using AIM technology around the world.
AIM stands out among other music technology research groups because of its pedagogy. While other music technology groups may cater primarily to graduate students and professionals, our group is open to all Purdue students of any major and experience level.
We hope that by helping any student interested in music technology/machine learning learn to work with these technologies, we can make a difference in these students' lives while simultaneously encouraging the adoption of music technology.
