Semester II, AY2015/2016 (Fridays 6.30 pm - 8.30 pm, SR2 (COM1 #02-04))

Last update: Friday, 18-Mar-2016 20:17:40 SGT

Table of Contents

• Announcements (last updated 16-Jan-2016)
• General Information
• Brief Course Outline
• Course Material
• Assignment Information
• Project Information
• Class Schedule

General Information

Lecturer: Roger Zimmermann

Aims and Objectives:

This course lays the foundation for graduate students to build advanced multimedia computing applications comprising of images, videos, and audio. The module covers the important multimedia computing methods by presenting a comprehensive coverage of the underlying content processing, content transformation and resource optimization techniques in a variety of systems such as multimedia information retrieval, conferencing, surveillance and security. By considering the research issues in the multimedia systems areas, it will also prepare the student in formulating novel approaches for future multimedia computing applications.

Brief Description:

By the end of the course, the students will be familiar with the common computing fundamentals employed in a variety of multimedia applications such as: content-based multimedia retrieval, summarization, surveillance, multimedia security and computational advertisement. The students will be exposed to the core techniques and algorithms spanning across the common and emerging multimedia applications. They will have experience in applying these techniques to novel situations and will be able to do analytical as well as empirical performance evaluation of the particular technique in the overall application context.

Grading information:

Survey Paper: 20%
Assignment: 20%
Project: 60%

Teaching Assistant:

• Wang Yuhui

Pre-requisites:

• Desirable: CS5240 Theoretical Foundations of Multimedia

Office consultation hours:

• Roger Zimmermann   (by appointment)

<-- Table of Contents

Brief Course Outline

* Introduction to Multimedia Computing (2 hrs), [Slides]
   Motivation; Fundamentals of Multimedia Computing; Image, Video and Audio Compression Overview
* Content-based Retrieval (6 hrs)
   Image Retrieval; Video Retrival; Tagging
* Multimedia Content Processing (4 hrs)
   Multimodal Data Fusion; Visual Saliency & Experiential Sampling
* Multimedia Surveillance (2 hrs)
   Background Modeling; Object Tracking; Use of Multiple Sensors; Decision-theoretic Methods
* Multimedia Summarization (2 hrs)
   Video Summarization; Multimedia Simplification
* Multimedia Data Mining (2 hrs)
   Probabilitic Concepts; Image/Video Mining; Concept Mining
* Multimedia Security (2 hrs)
   Watermarking; Forensics
* Computational Multimedia Advertisement (2 hrs)
   Computational Advertisement Framework; Multimedia Analysis for Ad Placement
* Current Issues & Trends (2 hrs)

<-- Table of Contents

Course Material

<-- Table of Contents

Assignment Information

• Survey Paper is due on Wednesday, February 10, 2016.
• Assignment is due on Monday, March 7, 2016.

<-- Table of Contents

Project Information

The details of the project will be posted here:

• Project proposal is due on Friday, February 5, 2016.
• The interim project report is due on Friday, March 11, 2016.
• The final project submission is due on Monday, April 11, 2016.

To give you some ideas, past projects have included the following (titles):

• Implementing Secure Randomized Image Watermarking Based on Singular Value Decomposition
• Facial Recognition on a Robotic Platform for Human-Robot Interaction
• Content Based Image Retrieval System using Texture and Modified Block Truncation Coding and Bag of Visual Words
• Video Reflection Removal by Exploring Temporal Structure
• Application of Stacked Denoising Autoencoders for Classification of Retina Images
• Application with Enhanced XOR-Base Image Encryption
• Face Verification using Joint Bayesian Classifier and HighDimLBP Features
• Augmented Illumination in Reducing HAR Geometric Consistency
• etc.

<-- Table of Contents

Class Schedule

15^th Jan (Week 1): Lecture 1: Introduction to Multimedia Computing  

• Reading: Wallace's JPEG Tutorial; MPEG Paper.

22^rd Jan (Week 2): Lecture 2: Content Based Retrieval I  

• Reading: Content-based Image Retrieval at the End of the Early Years; Application Potential of Multimedia Information Retrieval.

29^th Jan (Week 3): Lecture 3: Content Based Retrieval II 

• Reading: Automatic Image Annotation and Retrieval using Cross-Media Relevance Models.

5^th Feb (Week 4): Lecture 4: Content-based Retrieval III 

• Reading: Applications of Video Content-Analysis and Retrieval; The Unreasonable Effectiveness of Data.

12^th Feb (Week 5 - Lunar New Year): Lecture 5: Multimedia Content Processing I 

• Reading: Information Assimilation Framework for Event Detection in Multimedia Surveillance Systems, Multimodal Fusion for Multimedia Analysis: A Survey.

19^th Feb (Week 6): Lecture 6: Multimedia Content Processing II  

• Reading: Experiential Sampling in Multimedia Systems.

4^th Mar (Week 7): Lecture 7: Multimedia Surveillance I  

• Reading: Adaptive Background Mixture Models for Real-time Tracking. Tracking Paper, Tracking with Kalman Filtering

11^th Mar (Week 8): Lecture 8: Multimedia Summarization  

• Reading: Video Summarization using R-Sequences; Compressed Domain Video Summarization; Semantic Video Summarization; Multimedia Simplification.

18^th Mar (Week 9): Lecture 9: Multimedia Data Mining  

• Reading: Multimedia Data Mining: State of the Art and Challenges; Probabilistic Temporal Multimedia Data Mining.

25^th Mar (Week 10 - Good Friday - No Lecture)  

1^st Apr (Week 11) Lecture 9: Multimedia Security  

• Reading: Content-based Watermarking of Images, Compressed Domain Scrambler/Descrambler for Digital Video.

8^th Apr (Week 12): Lecture 11: Computational Multimedia Advertisement  

• Reading: CAVVA: Computational Affective Video-in-Video Advertising.

15^th Apr (Week 13): Lecture 12: Current Issues & Trends  

<-- Table of Contents