AIM (Artificial Intelligence for Musicians) is a Purdue music technology research group, whose aim is to create reactive, human-like systems which support musicians during their practice sessions and performances.
Some of AIM's projects are supported by a National Science Foundation grant.
We are looking for motivated graduate or undergraduate students to join our team. Click here for info.
Evaluator is an app that aims to help musicians practice more effectively. It utilizes computer vision and YOLO localization techniques to help musicians track, analyze, and improve their posture. It also uses spectrogram analysis and multi-modal transformers to help the musicians identify their mistakes in music and correct them.
Drawing by Cecilia Ines Sanchez.
Companion is an app that not only plays along with a human player during a chamber music piece, but actively responds to their playing habits and voice commands like a real human would. The project involves machine learning and filtering/DSP algorithms to analyze and edit sound quickly and accurately and utilizes small NLP language models for voice command implementation.
Drawing by Cecilia Ines Sanchez.
Mus2Vid is a real-time art project that uses diffusion models to generate video depictions in response to classical music. It uses recurrent and transformer networks to analyze input audio and estimate its emotion and genre qualities, which are converted into text and fed to a text-to-image diffusion model to generate images.
As the name suggests, Robot Cello is a project about using reinforcement learning to teach a robot arm to play cello. The project is currently in its survey phase but is currently investigating using motion capture technology to get training data for an RL model.
We partner with the Purdue Envision Center to collect motion data for our robot arm to train on. On the left is a video of Prof. Yun playing cello while wearing a motion-capture rig.
Our projects often span most or all of these areas, as they are all important to making effective, human-like music technology.
How can Companion utilize machine learning and filtering to resynthesize string instrument articulations on-the-fly?
What are the most effective methods for Companion, Evaluator, and Mus2Vid to follow a musician's playing in reference to a score, and play along to match?
How can Mus2Vid analyze emotion of classical music in real-time and utilize it to generate real-time video accompaniments?
What musical features extracted from various media, such as tempo, key, genre, notes, are useful to music performance technology, and how can we extract such features?
How can our apps be designed in ways that are human-like and natural for humans to interact with?
How can we ensure that our users actually utilize and enjoy the apps we develop?
Our team comes from a wide variety of backgrounds, including the Purdue Colleges of Engineering, Science, Liberal Arts, Management, and more. We have a mix of Purdue professors, graduate students, and undergraduate students leading our projects and research efforts.
Kristen Yeon-Ji Yun is a clinical associate professor in the Department of Music in the Patti and Rusty Rueff School of Design, Art, and Performance at Purdue University. She is active as a soloist, chamber musician, musical scholar, and clinician. Her recent CD “Summerland” has excellent reviews from New Classics UK, American Record Guide, and was broadcast nationwide by radio stations such as WQXR, WCNY, WBAA, and NPR Sonatas and Soundscapes. She started this research group due to struggles she personally experienced while practicing and performing cello.
With an academic concentration on artificial intelligence and machine learning, and over 12 years of experience as a classical pianist, I am a past and present leader of two research projects analyzing music with AI. I led a team investigating audio input for image diffusion models, translating classical music into visual art in real time. Now, I lead a team developing an app to analyze and critique string musicians’ sound and posture using multiple streams of input – audio and video of performances, as well as digitized sheet music. Outside of research and schoolwork, I spend his time cooking, camping, and rock climbing.
Tim is the leader of the Companion project, as well as an active contributor to Mus2Vid. His interests lie primarily in building software systems that exhibit emotion in human-like ways, usually through the lens of music technology. In his free time, Tim composes and produces pop/electronic music, (badly) plays the piano, and enjoys casually travelling and hiking.
Kareena is a leader of the Mus2Vid project. She has a genuine passion for learning and exploring new ideas. Whether it's diving into machine learning and deep neural networks or staying updated with industry trends, she is always eager to expand my knowledge and skills. Outside of academics, Kareena likes to explore her creative endeavors such as writing songs and playing the piano. She also has a passion for the outdoors, and enjoys kayaking and hiking.
Haichang (Charles) Li (李海畅) is a leader of the Mus2Vid project. He is currently interested in Human-AI Collaboration, specifically at the intersection of AGI and HCI (in especial creative works like Music & modeling). His ideal role is to be the link and bridge between external observers and designers in this process. He hopes to explore how AI can better help human beings to achieve benign coexistence, such as in the fields of multi-modal a11y, that is, to help people who need help more first and to explore the impact of AI on the world.
Hi! I am Sam, and I'm the leader of the Robot Cello project. I have experience with C/C++ systems programming, Java programming, data engineering using Python, database and SQL, and various other aspects of computer science. I am also minoring in psychology to understand better about human minds and the connection with AI.
In my free time, I enjoy playing cello, reading, and listening to classical music :)
Brian was a founding member of Purdue AIM. He received his B.S.CmpE. from the Purdue College of Engineering in December 2023.
Here are some of the ways AIM ensures that it is integrated with both Purdue's local community and the global community.
AIM has an associated Vertically Integrated Project, which enables research experiences for Purdue undergraduates.
Music technology is a field with very multidisciplinary problems and solutions. As such, we draw from a wide variety of departments for our talent, including Music, Electrical and Computer Engineering, Computer Science, Technology, Art, Design, and Management.
AIM actively runs user studies on the tools that are developed to ensure that users can figure out how to use them and enjoy them.
AIM members have given speeches, presentations, and concerts about and using AIM technology around the world.
AIM stands out among other music technology research groups because of its pedagogy. While other music technology groups may cater primarily to graduate students and professionals, our group is open to all Purdue students of any major and experience level.
We hope that by helping any student interested in music technology/machine learning learn to work with these technologies, we can make a difference in these students' lives while simultaneously encouraging the adoption of music technology.
