InstructFX2FX
CNMAT Lab, UC Berkeley · Research Assistant
- Working on instruction-guided audio effect transfer using language-conditioned models
- Developing methods for controllable audio transformation through natural language instructions
Song-Ze (Jimmy) Yu
An engineer & a musician
CS @ UC Berkeley · ML Researcher · Pianist (24 solo concerts)
Download CVAbout
I'm a Computer Science student at UC Berkeley (exchange from National Tsing Hua University), passionate about the intersection of machine learning and music technology. My goal is to build Audio Language Models for music — inspired by the rapid progress of Vision-Language Models, I believe similar breakthroughs are possible in the audio domain.
Beyond engineering, I'm a classical pianist with 24 solo concerts and over 70 TV appearances across Asia. This dual perspective as both an engineer and a musician drives my research and creative work.
I'm also a 4× hackathon winner. One of my projects, ParkFlow, was integrated into Taipei's TownPass super-app, reaching over 3 million downloads.
Research
My research focuses on Audio Language Models and music information retrieval, aiming to bring the power of modern deep learning to music understanding and generation.
VTR Model
AHG Music Lab, NTHU · Research Assistant
- Developed a Voice Timbre Replacement model for real-time vocal transformation
- Built a JUCE-based VST plugin (VTR-SmartEQ) for practical music production use
Experience
Berkeleytime
ML Pod Lead & Full-Stack Engineer
- Leading the ML pod to build intelligent course recommendation and scheduling features
- Contributing full-stack with TypeScript, GraphQL, React, and MongoDB
- Collaborating with a team of engineers to serve 40,000+ Berkeley students
Positive Grid
Machine Learning Intern
- Applied ML techniques to guitar amp modeling and audio effect processing
- Worked on neural network architectures for real-time audio inference
- Collaborated with audio DSP engineers to integrate ML models into production plugins
Projects & Awards
Awards
Oct 2025
Claude × a16z × Berkeley M.E.T. Makeathon — Best Design
Oct 2024
Meichu Hackathon 2024 — 2nd Place, Google
Sep 2024
Taipei Hackathon 2024 — 3rd Place → Integrated into TownPass (3M+ downloads)
Jul 2024
SITCON Hackathon 2024 — 1st Place
Selected Projects
Claude Code Remote
An open-source tool that gained widespread attention with 126K+ views on X. Enables remote development workflows with Claude.
View on GitHub →VTR-SmartEQ
A VST plugin for voice timbre replacement, enabling real-time vocal transformation in music production environments.
Let's Connect
I'm always open to discussing research, music, or collaboration opportunities.
vaclis@berkeley.edu