Robotic Musician
A workshop exploring the robotic technologies to play musical instruments.
Time and Location: TBD
Workshop Summary
This workshop will explore the technologies that can enable robots to play musical instruments. Virtuosity of musical instruments is considered one of the most challenging and elevated forms of human expression. Despite decades of progress in robotics technology, today's robots are still not ready to play musical instruments at the same level as professional musicians. Playing musical instruments requires coordination of movement and precise timing with appropriate amounts of forces. For different instruments, different body parts are required, such as hands, mouth, or feet. The robots must possess fine motor skills coupled with advanced intelligence and can integrate multimodal sensing, real-time response, and correctly articulating musical notes. This workshop will provide a snapshot of the state of the art in robotic technologies for playing musical instruments. The workshop will feature invited speakers that are recognized experts in this field. The authors of selected papers will present their recent work. A panel discussion will explore research directions for the community. The workshop will conclude with a poster session and demonstrations.
Organizers
Yung-Hsiang Lu is a professor in the Elmore Family School of Electrical and Computer Engineering at Purdue University. He is a fellow of the IEEE and distinguished scientist of the ACM. He has published papers on topics about computer vision and machine learning in venues such as AI Magazine, Nature Machine Learning, and Computer. He is one of the editors of the book "Low-Power Computer Vision: Improve the Efficiency of Artificial Intelligence" (ISBN 9780367744700, 2022 by Chapman & Hall).
Kristen Yeon-Ji Yun is a clinical associate professor in the Department of Music in the Patti and Rusty Rueff School of Design, Art, and Performance at Purdue University. She is the Principal Investigator of a research project "Artificial Intelligence Technology for Future Music Performers" (US National Science Foundation, IIS 2326198). She is an active soloist, chamber musician, musical scholar, and clinician. has toured many countries including Malaysia, Thailand, Germany, Mexico, Japan, China, Hong-Kong, Spain, France, Italy, Taiwan, and South Korea giving a series of successful concerts and master classes.
Zhiyao Duan is the President of the International Society for Music Information Retrieval (ISMIR), and a member of the Audio and Acoustic Signal Processing (AASP) technical committee of the IEEE Signal Processing Society. He co-organizes the inaugural Singing Voice Deepfake Detection (SVDD) Challenge at the IEEE Workshop on Spoken Language Technology (SLT) and ISMIR in 2024. He served as a scientific program co-chair of the ISMIR conference in 2021 and the publications chair in 2017. He served as the chair of the North East Music Informatics Special Interest Group (NEMISIG) workshop in 2017, and session chairs in ISMIR, ICASSP and Interspeech. He is a senior area editor of IEEE Signal Processing Letters and an associate editor of IEEE Open Journal of Signal Processing. He was a guest editor of Transactions of International Society for Music Information Retrieval (TISMIR).
Guoyu Lu is an Assistant Professor in the School of Electrical and Computer Engineering at University of Georgia. Before joining UGA, he was an assistant professor at Rochester Institute of Technology, a research scientist on autonomous driving at Ford, and computer vision engineer at Disney ESPN Advanced Technology Group. Guoyu Lu finished his PhD and MS at University of Delaware. Before studying at UD, he was enrolled in the European Master in Informatics (EuMI) Erasmus Mundus program. Lu obtained his master degree in Computer Science at University of Trento, and earned a master degree in Media Informatics at RWTH Aachen University. Lu also took an appointment as a visiting scholar at Auckland University of Technology, and was a research intern at Siemens Corporate Research in Princeton and Bosch Research in Palo Alto. Lu earned his B.S. in Software Engineering at Nanjing University of Posts & Telecommunications, with a minor in Business Administration and Management. Guoyu Lu has been PI for projects funded by NSF (CSSI, CRII, Core SHF), USDA, Ford, GM, Qualcomm, Tencent, Mackinac, etc. Guoyu Lu received the USDA New Investigator Award, Ford URP Award, Tencent Rhino-bird Young Faculty Award, Frank A. Pehrson Award, and Erasmus Mundas Scholarship. Dr. Lu’s research interests are in computer vision, AI, deep / machine learning, robotics, multimedia, and AI-based infrastructure systems.