Song-Ze Yu performing a solo recital at a grand piano

Before research

I performed 24 solo recitals and 70+ TV shows across China and Taiwan. I still perform, compose, and produce, and that musician’s perspective shapes both the research directions I pursue and the approach I take.

Music CVYouTube (7K subscribers)Instagram

Song-Ze Yu at the piano as a child, showing an original composition to Jay Chou
Showing my composition to Jay Chou.

About

I’m a Computer Science student at UC Berkeley and NTHU. I work with Berkeley AI Research (BAIR) under Prof. Trevor Darrell and David M. Chan, and with the NTU Music and AI Lab under Prof. Yi-Hsuan (Eric) Yang. Previously, I worked with Center for New Music & Audio Technologies (CNMAT) under Prof. Carmine-Emanuele Cella.

My research focuses on grounded music understanding in large audio-language models and controllability for musicians in music and audio effects generation. My long-term goal is to bring music models toward artificial music intelligence, offering musicians new interfaces to directly discuss, collaborate, and interact with machines.

News

Publications * equal contribution

Music Understanding in Audio-Language Models

  1. PitchBench: Measuring Pitch Hearing in Audio-Language Models

    Milan Liessens Dujardin*, Song-Ze Yu*, Craver Corbyn Thomas-Smith, David M. Chan, Karina Nguyen

    NeurIPS 2026, Evaluations & DatasetsPaperDatasetCode

  2. UniFront: Revisiting Time-Frequency Frontends for Unified Music and Speech Modeling

    Song-Ze Yu, David M. Chan

    Under review, 2026

  3. Diagnosing Music Understanding in Audio-Language Models: Encoder-Decoder Probing and Task Adaptation

    Jun-You Wang, Song-Ze Yu, Xuanjun Chen, Chih-Kai Yang, Ke-Han Lu, Sung-Feng Huang

    Under review, 2026

  4. MuSP-Bench: Advanced Multimodal Benchmarking of Music Understanding across Score and Performance

    Milan Liessens Dujardin, Song-Ze Yu, Kevin Miao

    ISMIR 2026, Late Breaking DemoPaperDataset

  5. Unifying Score and Performance for Fine-Grained Music Understanding in Audio-Language Models

    Milan Liessens Dujardin, Song-Ze Yu, Kevin Miao

    Under review, 2026PaperDemo

Controllability for musicians

  1. InstructFX2FX: A Multi-Turn Text-to-Effect System for Sequential Audio Effect Refinement

    Song-Ze Yu, Milan Liessens Dujardin, Yuxuan Cai, Wantong Zhang, Brian Cruz, Jeremy Wagner, Carmine-Emanuele Cella

    DAFx 2026, DemoPaperDemoCode

  2. MIDI-REPA: Improving Text-to-Music Generation with Symbolic Representation Alignment

    Song-Ze Yu, Fang-Duo Tsai, Yi-Hsuan Yang

    Under review, 2026PaperProjectCode

HCI & human simulation

  1. Anamnesis: An Open-Source Platform for Large-Scale Backstory-Conditioned Survey Simulation

    Song-Ze Yu, Joseph Suh, Serina Chang, David M. Chan

    EMNLP 2026, System DemonstrationsPaperDemo

Experience

Full experience, awards & projects →