SMORPH: Playable Sound Morphing
with Diffusion Models

abstract

Sound morphing—generating intermediate sounds that transition from one sonic identity to another—is a powerful tool for sound design and musical exploration. A musician may want a snare hit to shift from metallic to wooden, or a melodic phrase to move from brass to strings, while keeping rhythm, pitch, and dynamics stable as only the sound’s character changes. Existing diffusion-based morphing approaches entangle temporal structure and timbral identity, offering no explicit mechanism to hold one fixed while transforming the other. We present SMORPH, a training-free framework that preserves how a sound behaves over time while transforming what the sound is, enabling three structure-preserving morphing modes: prompt-to-prompt, audio-to-prompt, and audio-to-audio. When precomputed and exposed as semantic sliders, these morph trajectories become playable interfaces offering continuous, real-time timbral control and instrumental interaction. Evaluations across music and environmental sound datasets show that explicit structural decomposition improves structural adherence and timbral progression without sacrificing morph quality, and an instrument-design case study demonstrates semantic sliders as an expressive interface for musical exploration and performance.