Multimedia Data Mining

Advances in multimedia data acquisition and storage technology have led to the growth of very large multimedia databases. Analyzing this huge amount of multimedia data to discover useful knowledge is a challenging problem. This challenge has opened the opportunity for research in Multimedia Data Mining (MDM). Multimedia data mining can be defined as the process of finding interesting patterns from media data such as audio, video, image and text that are not ordinarily accessible by basic queries and associated results. The motivation of doing MDM is to use the discovered patterns to improve decision making. MDM has therefore attracted significant research efforts in developing methods and tools to organize, manage, search and do domain  specific tasks for data from domains such as surveillance, meetings, broadcast news, sports, archives, movies, medical data, as well as personal and online media collections. Existing multimedia data mining techniques assume that the obtained events or concepts from event detectors are accurate. However, in reality detectors label the events/concepts from different modalities with a certain confidence measure over a time-interval. Therefore, it is important to consider the uncertainties associated with the event sequences into the process of multimedia data mining. We consider for the first time Probabilistic Temporal Multimedia (PTM) data to discover meaningful patterns. As the existing data mining techniques cannot work on such real data, we have developed a novel framework for performing multimedia data mining on probabilistic temporal multimedia data.

In this research work, we aim to design and develop a novel framework for performing multimedia data mining on probabilistic temporal multimedia data. Specifically, we would investigate the following:

• Consideration of confidence value for finding granular level frequent temporal event sequences. For example, if traditional sequence pattern mining method discover a single semantic event sequence1 = {Event(A) “overlap” Event(B)}, our method can discover more than one semantic event sequences for similar dataset, sequence1 = {Event(A with 0.5 confidence) “overlap” Event(B with 0.7 confidence)}, sequence2 = {Event(A with 0.3 confidence) “overlap” Event(B with 0.2 confidence)} and sequence3 ={Event(A with 0.8 confidence) “overlap” Event(B with 0.3 confidence)}.
• Use of confidence fusion to resolve the redundancy among the events detected from different modalities considering their cross-modal correlation for data mining purpose. It is an important data integration step.
• Conceptual clustering of multimedia data using their confidence value distribution over the time. The method discovers the cluster of semantically similar multimedia data.
• Classification using frequent event sequence confidence correlation. Suspicious events are detected from their preceding frequent event sequences.