Background informationReader of the course:
Readersmall.pdf (low resolution: 30MB)
Readerlarge.pdf (high resolution: 120MB)
Excercises of the course:
Slides of the course:
ContentsWith the growth of the Internet and developments in imaging technology, very large digital image and video archives have been created and used in numerous applications. Together with the increase in the number of pictorial archives, demands are also growing for methodologies and techniques to store and retrieve pictorial entities from large image archives.
In this course a broad range of techniques are studied to access multimedia information including multimedia features (synonyms for text, color and shape invariants for images), multimedia information space modeling (logic model, vector space model, statistical model), indexing (kd-trees, inverted file), learning and classification (nearest neighbor, neural network), user interaction (active learning), visualization and presentation techniques.
Teaching formatLectures, workshops and lab sessions
ECTS Points6 ECTS
ExamsThere will be 1 written exam and 1 assignment. The written exam is 50% of the final result. The assignment will be the other 50%.
The assignment is:
Lab Session AssignmentThe goal of this project is to study on computational methods to track objects in video sequences. The visual tracking of objects in image sequences is an important and challenging problem with many practical applications such as tracking of humans at various scales in surveillance, video conferencing, and man-machine interaction, tracking cars and air-planes in traffic control, and particles in medicine. In this project, we focus on the following application to bootstrap a wide range of sequel applications: Tracking of players/objects in sport activities (i.e. soccer). For these applications, the interest of visual tracking needs to address changes in visual appearance as well as occlusion. We focus on the following two main problems in object tracking:
* How to track objects, while the illumination and appearance of the object changes considerably?
* How to keep track of track objects while they are occluded?
PrerequisitesFor this project a tracking application in Matlab will be developed. This requires of course the ability to write programs in Matlab. However the first Lab session will be an introduction to Matlab. During the next sessions you will learn how to translate formula's and algorithms from papers to working Matlab programs. Handouts and example code will be made available on the Lab website, during the course.
GoalThe project studies on computational techniques to track objects based on histogram comparison. To this end, the tracking algorithm will focus on a model histogram of the object to be tracked. For each (current) frame location within a specific window (e.g. 15x15), a histogram is built and compared with the model histogram. The goal is to find a maximum value (best match) within this 15x15 neighbourhood. The search for a maximum is by exhaustive search or by the mean shift algorithm. We will focus on color invariant models to achieve robustness against illumination and changes in object appearance.
Problem Solving StepsFirst, we study on existing techniques to track objects by histogram matching (e.g. D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-Based Object Tracking", IEEE trans. on Pattern Recognition and Machine Intelligence, May 2003, vol. 25, number 5, pp. 564-578, 2003, http://www.caip.rutgers.edu/~comanici/Papers/KernelTracking.pdf). Color invariant models are analyzed which are invariant to illumination conditions (Th. Gevers, Color in Image Search Engines Survey on color for image retrieval from Multimedia Search, ed. M. Lew, Springer Verlag, January, 2001. http://carol.science.uva.nl/~gevers/pub/survey_color.pdf). Color/texture histograms are built and incorporated in the tracking mechanism. The mean-shift algorithm will be used as the tracking engine. The following issues will be addressed:
* Color models. Which color model is suited to obtain robustness against illumination conditions.
* Representation. Color histograms (homogeneous objects) wrt texture/correlograms (textured objects).
* Histogram matching. Bhattacharyya coefficient wrt Euclidean distance.
* Computational complexity. Exhaustive search and mean-shift algorithm.
* Accuracy. Incorporation of background and (object) scale information.
A complete evaluation will also include a video from another domain (other than soccer) to test the generality of the approach. Students are expected to bring their own video data (home-video, from DVD, from the internet).
Lab OverviewThe lab will be held approximately thirteen sessions of two hours each. The structure of the lab is as follows:
Introduction to Matlab
Image processing in Matlab
Advanced histogram items
Brute force object tracking
Kernel based tracking
Finishing and evaluating the tracker
ExaminationEvaluation of your work will be based on the produced implementation (25%), and the report written (75%). In turn, the result of this course constitutes 50% of your final result, just like the result of the final paper examination.
The report to be written must be about 5-7 pages in length, with a reasonable amount of text on each page. It should contain information about design choices, implementation issues, and system performance (how good are the results, are there still possibilities for optimization, etc). Papers must be written in English.