Automatic Music Transcription

Fall 2024 - present

Automatic Music Transcription is a project focused on streamlining audio-to-MIDI transcription for musicians and educators, with applications in isolating sounds in noisy environments. We are conducting a systematic review of AMT models, examining their strengths and limitations with complex, multi-instrument music. We hosted a competition in April 2025, challenging participants to create accurate transcription models for classical music. Currently, we are building our own interpretable AMT model, as well as focusing on other niches such as using computer vision to generate guitar tablature and quantifying a music piece's "complexity" as inputs for future AMT models.

Evaluator

Fall 2023 - present

Evaluator is an app that aims to help musicians practice more effectively. It utilizes computer vision and YOLO localization techniques to help musicians track, analyze, and improve their posture. It also uses spectrogram analysis and multi-modal transformers to help the musicians identify their mistakes in music and correct them.

Companion

Fall 2023 - present

Companion is an app that not only plays along with a human player during a chamber music piece, but actively responds to their playing habits and voice commands like a real human would. The project involves machine learning and filtering/DSP algorithms to analyze and edit sound quickly and accurately and utilizes small NLP language models for voice command implementation.

Mus2Vid

Spring 2022 - present

Mus2Vid is a real-time art project that uses diffusion models to generate video depictions in response to classical music. It uses recurrent and transformer networks to analyze input audio and estimate its emotion and genre qualities, which are converted into text and fed to a text-to-image diffusion model to generate images.

• Learn more

Robot Cello

Spring 2024 - present

As the name suggests, Robot Cello is a project about using reinforcement learning to teach a robot arm to play cello. The project is currently in its survey phase but is currently investigating using motion capture technology to get training data for an RL model.

We partner with the Purdue Envision Center to collect motion data for our robot arm to train on. On the left is a video of Prof. Yun playing cello while wearing a motion-capture rig.

Meet the team!

Our team comes from a wide variety of backgrounds, including the Purdue Colleges of Engineering, Science, Liberal Arts, Management, and more. We have a mix of Purdue professors, graduate students, and undergraduate students leading our projects and research efforts.