Semester II, 2001-2002 (Wed 8am-10am, SR1)
Last update: Monday, 01-Apr-2002 13:04:27 SGT
Lecturer: Mohan S KankanhalliLectures: 26 Hours (Wednesday, 8am-10am, SR1)
Aims and Objectives:
This course introduces techniques for analysis, representation and retrieval of multimedia information. The media to be considered include image, audio and video. At the end of this course, the students should have the expertise and competence to design and implement retrieval software for multimedia data.
Multimedia data is unstructured and rich in content. Conventional database systems, which are designed to handle structured data and support exact match retrieval, are inadequate for this type of data. This course covers a broad class of retrieval techniques based on similarity-based retrieval. Similarity-based retrieval relies on "best-match" rather than "exact match" and uses techniques to compute the "similarities" between the query and information items. As the users' information needs are also fuzzy, an important characteristic for this class of retrieval techniques is its support for the iterative process of retrieval.
In this course, we will discuss various attributes characterizing the multimedia data. The attributes to be discussed are text, color, texture and shapes. For each attribute, we will discuss its representation scheme, similarity-based retrieval model, iterative refinement technique, and other representation and retrieval models. We will discuss the use of these attributes to retrieve images, audio and video. Finally, we will discuss a framework for multimedia information retrieval and directions of the future work.
Midterm Exam: 20%
Final Exam: 50%
Pre-requisites: CS3242 (Hypermedia Information Processing)
Office consultation hours: Wed 3-5pm (S17 #04-19)
I. Introduction to multimedia information retrieval 6 hrs
characteristics of MM data; similarity-based retrieval model; attributes of MM data; similarity measures; multimedia retrieval framework; relevance feedback; benchmarking of multimedia information systems
II. Color-based Retrieval 4 hrs
color models; histogram model; indexing and retrieval; relevance feedback; histogram refinement; color cluster technique
III. Texture-based Retrieval 2 hrs
texture models; statistical models; combined color-texture representation
IV. Shape-based Retrieval 2 hrs
shape matching; contour-based method (Fourier descriptors); region-based method (moment invariants)
V. Audio Retrieval 4 hrs
characteristics of audio data; spectrum analysis; pitch tracking; techniques for audio feature extraction, similarity matching and retrieval
VI. Video Retrieval 4 hrs
video segmentation in raw and compressed domain; key-frame extraction; video summarization and retrieval techniques
VII. Multimedia Retrieval Framework 2 hrs
multi-attribute query processing; knowledge-based methods
VIII. Multimedia Retrieval Trends 2 hrs
A copy of the slides & all the relevant material will be put in the coop.
Some Books: (buying them is not at all necessary)
1. R. Jain, R. Kasturi, B.G. Schunck (1995), Machine Vision, McGraw-Hill.
[One of the many basic reference texts on image processing]
2. B. Furht, S.W. Smoliar, H.J. Zhang (1995), Video and Image Processing
in Multimedia Systems, Kluwer, Boston.
[A reference text on multimedia in general]
3. J.K. Wu, M.S. Kankanhalli, J.H. Lim, D.Z. Hong (2000), Perspectives on
Content-based Multimedia Systems, Kluwer Academic Publishers, Boston.
Tutorial 1: Tutorial on Retrieval Models (05-02-2002)
Tutorial 2: Tutorial on Color-based Retrieval (19-02-2002)
Tutorial 3: Tutorial on Texture and Shape Retrieval (05-03-2002)
Tutorial 4: Tutorial on Audio Retrieval (19-03-2002)
Tutorial 5: Tutorial on Video Retrieval (02-04-2002)
9th January (Week 1): Introduction to MMIR
16th January (Week 2): MM Retrieval Framework I
23rd January (Week 3): MM Retrieval Framework II
Assignment 1: Image retrieval assignment (Due Mar 4)
30th January (Week 4): Color-based Retrieval I
6th February (Week 5): Color-based Retrieval II
16th February (Week 6): Texture-based Retrieval 8.00am - 10.00am @ SOC1 #04-TR7 Note: make-up class for February 13th
20th February (Week 7): Shape-based Retrieval
22nd February AM: Midterm Examination
TR9 (SOC1, 6th Floor) at 9.30am
This will be an open-book exam and it will be based on the material covered till February 6 (which is color-based retrieval).
27th February Semester Break
6th March (Week 8): Audio Retrieval I
Assignment 2: Audio assignment (Due Mar 25)
13th March (Week 9): Audio Retrieval II
20th March (Week 10): Video Retrieval I
Assignment 3: Video retrieval assignment (Due April 15)
27th March (Week 11): Video Retrieval II
3rd April (Week 12): Multi-attribute & Knowledge-based Retrieval
10th April (Week 13): Multimedia Retrieval Trends
27th April PM: Final Examination
Assignment 1 FAQ
Assignment 2 FAQ
Assignment 3 FAQ