Music Video Summarization

	Edward Wijaya
	Hendra Setiawan

Contents

	Multimedia Mining
	Multimedia Mining for Digital Library
	Music Video Summarization
	Conclusion

Multimedia Mining

	Summarization cast as Content Mining Problem – different with typical data mining
	Multimedia Mining – Pattern Discovery

Multimedia Mining for Digital Library

	Growing number of multimedia material in Digital Library catered by Content-based Retrieval
	Good summary is good candidate to facilitate user’s relevant judgement
	Good summary serve better indexing
	Accelerate the information seeking process (HCI aspects in Digital Libraries) for multimedia materials.

Video Summarization

	Strong visual-audio synchronization
		Audio-centric
		Visual-centric
	Weak visual-audio synchronization
		Summarize each track
		Alignment process

Music Video Summarization

	Weak synchronization
	Audio part is the salient part
	Goal :
	Provides a natural and effective audio-visual content overview
	Maximized the coverage for both audio and visual contents without having to sacrifice either of them
	[Shao Xi, et.al. Automatically Generating Summaries for Musical Video, 2003]

Music Summarization

	Problem : How to generate a concise and informative content that best summarize the original content
	Challenge :
		Featureless sequences of bytes (Feature Extraction is very important) →MPEG-7 standard
		Need content understanding
	Good candidate : Non-trivial repetitive pattern

Visual Summarization

	Challenges :
	Require overall understanding of the content
	Problem :
	Identify duplicates and redundancies in a video sequences (Easy)
	Finding the summary that has minimum redundancy

Visual and Audio Alignment

	Criteria :
	Smooth and Natural summary
	Maximize the coverage for both audio and visual content

Conclusion

	Summarization as Content Mining problem
	Multimedia Mining for Digital Library
	Music Video Summarization