AudeoSynth: Music-Driven Video Montage

to appear at SIGGRAPH 2015

Zicheng Liao1

Yizhou Yu2

Bingchen Gong1

Lechao Cheng1
1Zhejiang University

2The University of Hong Kong

Accompanying videos of the paper.


We propose music-driven video montage, a media format that offers a pleasant way to browse or summarize video clips collected from various occasions, including gatherings and adventures. In musicdriven video montage, the music drives the composition of the video content. According to musical movement and beats, video clips are organized to form a montage that "visually reflects the experiential properties of the music". Nonetheless, it takes enormous manual work and artistic expertise to create it. In this paper, we develop a framework for automatically generating music-driven video montages. The input is a set of video clips and a piece of background music. By analyzing the music and video content, our system extracts carefully designed temporal features from the input, and casts the synthesis problem as an optimization and solves the parameters through Markov Chain Monte Carlo sampling. The output is a video montage whose visual activities are cut and synchronized with the rhythm of the music, rendering a symphony of audio-visual resonance.


    title={audeosynth: Music-Driven Video Montage},
    author={Liao, Zicheng and Yu, Yizhou and Gong, Bingchen and Cheng, Lechao},
    journal = {ACM Trans. Graph. (SIGGRAPH) },
    volume = {34},
    number = {4},
    year = {2015},