Experiential Sampling in Multimedia Systems

Content

Background - Motivation for this work

Framework - Overview of our solution: a sampling based framework

Applications - Describes some applications such as automatic camera control, face detection video content representation and multiple-camera video surveillance.

Experimental Results - Some experimental results have been shown.

Test Clips - Some of the test video clips have been provided.

Experiential Systems - This is Ramesh Jain's website. It provides the background about experiential computing.

For the complete set of experimental results and details about the experiential sampling technique, please refer to our publications.

People

Mohan S. Kankanhalli
School of Computing, National University of Singapore, Singapore, mohan@comp.nus.edu.sg

Jun Wang
Department of Computer Science, University College London, UK, j.wang@adastral.ucl.ac.uk

Ramesh Jain
Bren School of Information and Computer Sciences, University of California, Irvine, jain@ics.uci.edu

Publications

First paper about experiential sampling:
[1] J. Wang and M. S. Kankanhalli, Experience-based Sampling for Multimedia Analysis, in Proc. of ACM Multimedia 2003. (Short paper), Berkeley, pp. 319-322, November, 2003. [pdf]

Multi-camera video surveillance by the experiential sampling technique:
[2] J. Wang, M. S. Kankanhalli, W-Q. Yan, and R. Jain, Experiential Sampling for Video Surveillance, in Proc. 1st ACM Int. Workshop on Video Surveillance, Berkeley, 2003. [pdf]

Automatic camera adjustment using the experiential sampling technique:

[3] J. Wang, W-Q. Yan, M. S. Kankanhalli, R. Jain, and M. J. T. Reinders, Adaptive Monitoring for Video Surveillance, in The Fourth IEEE Pacific-Rim Conference on Multimedia (PCM 2003), Singapore, 2003. [pdf]

Video adaptation via the experiential sampling technique:
[4] J. Wang. M. J. T. Reinders, R. L. Lagendijk, J. Lindenberg, and M. S. Kankanhalli, Video Content Representation on Tiny devices in Proc. IEEE International Conference on Multimedia and Expo. Taipei, July 2004. [pdf]

Complete formulation of the experiential sampling technique for multiple data streams environments:
[5] M.S. Kankanhalli, J. Wang, R. Jain, "Experiential Sampling in Multimedia Systems,", IEEE Transactions on Multimedia, 2006 (to appear). [pdf]

Objective

In this research, we would like to articulate the following goal for the multimedia systems:

"In an experiential computing environment, the system should sense the data (mutiple data streams) from the environment. Based on the observations and experiences, the system should collate the relevant data and information of interest related to the task. Thus, the system interacts naturally with all of the available data based on its interests in light of the past states in order to achieve its designed task."

What is "Experiential Sampling"?

The process of identifying the most relevant data among the available streams in the multimedia system at a given instant to utilize for interpretation to refine the current model of the environment.