ISIS home

Intelligent Multimedia Systems, Master AI, 2009


Background information

Reader of the course:
pdf format Readersmall.pdf (low resolution: 30MB)
pdf format Readerlarge.pdf (high resolution: 120MB)

Excercises of the course:
pdf format Part I
pdf format Part II

Slides of the course:
pdf format lecture1_2009.pdf
pdf format lecture2_2009.pdf
pdf format lecture3_2009.pdf
pdf format lecture4_2009.pdf
pdf format lecture5_2009.pdf
pdf format lecture6_2009.pdf
pdf format lecture7_2009.pdf
pdf format lecture9_2009.pdf
pdf format lecture10_2009.pdf
pdf format lecture11_2009.pdf
pdf format lecture12_2009.pdf
pdf format lecture13_2009.pdf
pdf format lecture14_2009.pdf
pdf format lecture15_2009.pdf
pdf format Pascal_VOC_2008_summary.pdf


Theo Gevers


Master A.I.


With the growth of the Internet and developments in imaging technology, very large digital image and video archives have been created and used in numerous applications. Together with the increase in the number of pictorial archives, demands are also growing for methodologies and techniques to store and retrieve pictorial entities from large image archives.

In this course a broad range of techniques are studied to access multimedia information including multimedia features (synonyms for text, color and shape invariants for images), multimedia information space modeling (logic model, vector space model, statistical model), indexing (kd-trees, inverted file), learning and classification (nearest neighbor, neural network), user interaction (active learning), visualization and presentation techniques.


I (1ab)

Teaching format

Lectures, workshops and lab sessions

ECTS Points



There will be 1 written exam and 1 assignment. The written exam is 50% of the final result. The assignment will be the other 50%.
The assignment is:
  • Lab session/practical course. The implementation of a visual tracking system.



Lab Session Assignment

The goal of this project is to study on computational methods to track objects in video sequences. The visual tracking of objects in image sequences is an important and challenging problem with many practical applications such as tracking of humans at various scales in surveillance, video conferencing, and man-machine interaction, tracking cars and air-planes in traffic control, and particles in medicine. In this project, we focus on the following application to bootstrap a wide range of sequel applications: Tracking of players/objects in sport activities (i.e. soccer). For these applications, the interest of visual tracking needs to address changes in visual appearance as well as occlusion. We focus on the following two main problems in object tracking:
* How to track objects, while the illumination and appearance of the object changes considerably?
* How to keep track of track objects while they are occluded?


For this project a tracking application in Matlab will be developed. This requires of course the ability to write programs in Matlab. However the first Lab session will be an introduction to Matlab. During the next sessions you will learn how to translate formula's and algorithms from papers to working Matlab programs. Handouts and example code will be made available on the Lab website, during the course.


The project studies on computational techniques to track objects based on histogram comparison. To this end, the tracking algorithm will focus on a model histogram of the object to be tracked. For each (current) frame location within a specific window (e.g. 15x15), a histogram is built and compared with the model histogram. The goal is to find a maximum value (best match) within this 15x15 neighbourhood. The search for a maximum is by exhaustive search or by the mean shift algorithm. We will focus on color invariant models to achieve robustness against illumination and changes in object appearance.

Problem Solving Steps

First, we study on existing techniques to track objects by histogram matching (e.g. D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-Based Object Tracking", IEEE trans. on Pattern Recognition and Machine Intelligence, May 2003, vol. 25, number 5, pp. 564-578, 2003, pdf format Color invariant models are analyzed which are invariant to illumination conditions (Th. Gevers, Color in Image Search Engines Survey on color for image retrieval from Multimedia Search, ed. M. Lew, Springer Verlag, January, 2001. pdf format Color/texture histograms are built and incorporated in the tracking mechanism. The mean-shift algorithm will be used as the tracking engine. The following issues will be addressed:
* Color models. Which color model is suited to obtain robustness against illumination conditions.
* Representation. Color histograms (homogeneous objects) wrt texture/correlograms (textured objects).
* Histogram matching. Bhattacharyya coefficient wrt Euclidean distance.
* Computational complexity. Exhaustive search and mean-shift algorithm.
* Accuracy. Incorporation of background and (object) scale information.
A complete evaluation will also include a video from another domain (other than soccer) to test the generality of the approach. Students are expected to bring their own video data (home-video, from DVD, from the internet).

Lab Overview

The lab will be held approximately thirteen sessions of two hours each. The structure of the lab is as follows:
Introduction to Matlab
Image processing in Matlab
Histogram creation
Advanced histogram items
Object localisation
Brute force object tracking
Kernel based tracking
Mean-shift tracking
Finishing and evaluating the tracker


Evaluation of your work will be based on the produced implementation (25%), and the report written (75%). In turn, the result of this course constitutes 50% of your final result, just like the result of the final paper examination.

The report to be written must be about 5-7 pages in length, with a reasonable amount of text on each page. It should contain information about design choices, implementation issues, and system performance (how good are the results, are there still possibilities for optimization, etc). Papers must be written in English.

Intelligent Sensory Information Systems
University of Amsterdam
The Netherlands
Copyright © 2006