Welcome to AIM.

AIM (Artificial Intelligence for Musicians) is a Purdue music technology research group, whose aim is to create reactive, human-like systems which support musicians during their practice sessions and performances.

Some of AIM's projects are supported by a National Science Foundation grant.

We are looking for motivated graduate or undergraduate students to join our team. Click here for info.

Evaluator

Fall 2023 - present

Evaluator is an app that aims to help musicians practice more effectively. It utilizes computer vision and YOLO localization techniques to help musicians track, analyze, and improve their posture. It also uses spectrogram analysis and multi-modal transformers to help the musicians identify their mistakes in music and correct them.

Drawing by Cecilia Ines Sanchez.

Companion

Fall 2023 - present

Companion is an app that not only plays along with a human player during a chamber music piece, but actively responds to their playing habits and voice commands like a real human would. The project involves machine learning and filtering/DSP algorithms to analyze and edit sound quickly and accurately and utilizes small NLP language models for voice command implementation.

Drawing by Cecilia Ines Sanchez.

Mus2Vid

Spring 2022 - present

Mus2Vid is a real-time art project that uses diffusion models to generate video depictions in response to classical music. It uses recurrent and transformer networks to analyze input audio and estimate its emotion and genre qualities, which are converted into text and fed to a text-to-image diffusion model to generate images.

Robot Cello

Spring 2024 - present

As the name suggests, Robot Cello is a project about using reinforcement learning to teach a robot arm to play cello. The project is currently in its survey phase but is currently investigating using motion capture technology to get training data for an RL model.

We partner with the Purdue Envision Center to collect motion data for our robot arm to train on. On the left is a video of Prof. Yun playing cello while wearing a motion-capture rig.

Research Areas + Questions

Our projects often span most or all of these areas, as they are all important to making effective, human-like music technology.

Generative Audio and DSP

How can Companion utilize machine learning and filtering to resynthesize string instrument articulations on-the-fly?

Beat Detection and Tempo Tracking

What are the most effective methods for Companion, Evaluator, and Mus2Vid to follow a musician's playing in reference to a score, and play along to match?

Emotion and Perception

How can Mus2Vid analyze emotion of classical music in real-time and utilize it to generate real-time video accompaniments?

Music Classification/Information Retrieval

What musical features extracted from various media, such as tempo, key, genre, notes, are useful to music performance technology, and how can we extract such features?

Human-Computer Interaction

How can our apps be designed in ways that are human-like and natural for humans to interact with?

User Studies + Deployments

How can we ensure that our users actually utilize and enjoy the apps we develop?

Meet the team!

Our team comes from a wide variety of backgrounds, including the Purdue Colleges of Engineering, Science, Liberal Arts, Management, and more. We have a mix of Purdue professors, graduate students, and undergraduate students leading our projects and research efforts.

Dr. Kristen Yeon-Ji Yun

Project Director

Clinical Associate Professor of Music

Kristen Yeon-Ji Yun is a clinical associate professor in the Department of Music in the Patti and Rusty Rueff School of Design, Art, and Performance at Purdue University. She is active as a soloist, chamber musician, musical scholar, and clinician. Her recent CD “Summerland” has excellent reviews from New Classics UK, American Record Guide, and was broadcast nationwide by radio stations such as WQXR, WCNY, WBAA, and NPR Sonatas and Soundscapes. She started this research group due to struggles she personally experienced while practicing and performing cello.

Benjamin "Ben" Shiue-Hal Chou

PhD Student and Lab Graduate Mentor

Ben is currently pursuing a PhD under the guidance of Dr. Yung-hsiang Lu. Since August 2023, he has been serving as a graduate mentor for the AIM program. His research focuses on the training and application of multimodal transformers to enhance the experiences and lives of musicians and music enthusiasts.

Haichang Li

Mus2Vid Project Lead

Haichang (Charles) Li (李海畅) is a leader of the Mus2Vid project. He is currently interested in Human-AI Collaboration, specifically at the intersection of AGI and HCI (in especial creative works like Music & modeling). His ideal role is to be the link and bridge between external observers and designers in this process. He hopes to explore how AI can better help human beings to achieve benign coexistence, such as in the fields of multi-modal a11y, that is, to help people who need help more first and to explore the impact of AI on the world.

Samantha Rose Sudhoff

Robot Cello Project Lead

Hi! I am Sam, and I'm the leader of the Robot Cello project. I have experience with C/C++ systems programming, Java programming, data engineering using Python, database and SQL, and various other aspects of computer science. I am also minoring in psychology to understand better about human minds and the connection with AI.
In my free time, I enjoy playing cello, reading, and listening to classical music :)

William Jiang

AIM Vision Lead (Evaluator and Companion)

William is the lead for all vision tasks on AIM's projects (specifically Evaluator). He is currently pursuing a triple BS in Computer Science, Math, and Statistics. In his free time, he enjoys playing chess and poker, along with finding good restaurants around Purdue.

Ryan Jordan

AIM Audio Lead (Evaluator and Companion)

Ryan is the lead for all audio tasks on AIM's projects. He is currently pursuing a BS in Computer Engineering. In his free time, Ryan loves traveling and exploring new places.

Edward "Eddie" Navarro

AIM Interaction Lead (Evaluator and Companion)

Eddie is the lead for voice commands and LLM control in AIM's projects. He is currently pursuing a BS in Computer Science with a minor in Economics. In his free time, Eddie loves listening to Chocolat Cadabra by Ado.

Tim Nadolsky

Mus2Vid and Companion Contributor

Tim is an active contributor to AIM's Mus2Vid and Companion projects. His interests lie primarily in building software systems that exhibit emotion in human-like ways, usually through the lens of music technology. In his free time, Tim composes and produces pop/electronic music, (badly) plays the piano, and enjoys casually travelling and hiking.

Brian Ng

Former Mus2Vid Project Lead + Lab Alumni

Brian was a founding member of Purdue AIM. He received his B.S.CmpE. from the Purdue College of Engineering in December 2023.

Dr. Victor Yingjie Chen

Professor of Computer Graphics Technology

Dr. Victor Yingjie Chen is a professor of Computer Graphics Technology. His research covers interdisciplinary domains of Computer Graphics and Human-Computer Interaction, such as Information Visualization, Visual Analytics, Virtual Reality, and AI in Computer Graphics. He seeks to design, model, and construct new forms of interaction in visualization and system design, by which the system can minimize its influence on design and analysis, and become a true free extension of human’s brain and hand.

Dr. Yung-Hsiang Lu

Professor of Electrical and Computer Engineering

Dr. Yung-Hsiang Lu is a professor of Electrical and Computer Engineering at Purdue University. He is a University Faculty Scholar of Purdue University. He is a fellow of the IEEE (Institute of Electrical and Electronics Engineers), distinguished visitor of the Computer Society, distinguished scientist and distinguished speaker of the ACM (Association for Computing Machinery). Dr. Lu is the inaugural director of Purdue’s John Martinson Engineering Entrepreneurial Center (2020-2022). In 2019, he received Outstanding VIP-Based Entrepreneur Award from the VIP (Vertically Integrated Projects) Consortium. His research areas include computer vision, embedded systems, cloud and mobile computing. Dr. Lu has advised 400 undergraduate students in research projects and taught more than 5,000 students in classrooms. He has advised multiple student teams winning business plan competitions; two teams of students started technology companies and raised more than $1.5M.

Dr. Cheryl Zhenyu Qian

Professor of Industrial Design, Rueff School of Design, Art, and Performance

Dr. Cheryl Zhenyu Qian is a full professor of Interaction Design in Industrial Design at Purdue University. Being a boundary crosser, Dr. Qian is interested in studying and developing cognitive systems to enrich knowledge, employing interdisciplinary research methodologies to improve the design quality, and adopting innovative technologies to accommodate user experience. Her current research is focused on 1) the harmonious integration of physical and virtual interactions in the user experience design, 2) adopting interaction design theories, tools, and evaluation methods to review, compare, guide, and enhance product design outcomes, and 3) employing innovative design thinking into the domain of visual analytics.

Dr. Mohammad Saifur Rahman

Professor of Management, Daniels School Chair in Management

Dr. Mohammad Saifur Rahman Professor Mohammad Saifur Rahman is the inaugural Daniels School Chair in Management and a Professor of Management at the Mitchell E. Daniels, Jr. School of Business, Purdue University. He was named one of the World's Top 40 Business School Professors Under 40 by Poets and Quants in 2017. His research primarily focuses on digitization economics, omnichannel retailing, innovations and inequality, and AI and decision making.
Professor Rahman has published in major journals including Management Science, Information Systems Research, and MIT Sloan Management Review. His papers have been accepted in several leading conferences, e.g., Workshop on Information Systems Economics (WISE), International Conference on Information Systems (ICIS), and Conference on Information Systems and Technology (CIST). Also, his research has been supported by multiple major Social Sciences and Humanities Research Council (SSHRC) grants.

Outreach

Here are some of the ways AIM ensures that it is integrated with both Purdue's local community and the global community.

Vertically Integrated Project

AIM has an associated Vertically Integrated Project, which enables research experiences for Purdue undergraduates.

Multidisciplinary Research

Music technology is a field with very multidisciplinary problems and solutions. As such, we draw from a wide variety of departments for our talent, including Music, Electrical and Computer Engineering, Computer Science, Technology, Art, Design, and Management.

User Studies

AIM actively runs user studies on the tools that are developed to ensure that users can figure out how to use them and enjoy them.

Presentations/Concerts

AIM members have given speeches, presentations, and concerts about and using AIM technology around the world.

Vertically Integrated Projects Team

AIM stands out among other music technology research groups because of its pedagogy. While other music technology groups may cater primarily to graduate students and professionals, our group is open to all Purdue students of any major and experience level.

We hope that by helping any student interested in music technology/machine learning learn to work with these technologies, we can make a difference in these students' lives while simultaneously encouraging the adoption of music technology.

Get in touch