Distributed multicamera surveillance

In this project we develop techniques for tracking multiple objects with multiple cameras. We focus on problems where the cameras are sparsely distributed over a wide area (think of an airport, shopping mall or a motorway). In this cases, the effective fields of view the cameras remain disjoint, therefore when an object (like a person or a car) leaves the view of some camera, we temporarily loose track of it. The key goal is re-identification of that object whenever it later on reappears in the view of some other camera. In this way we can recover the global trajectories of objects in the considered area. Interested? Try some of our demos.

In order to identify objects when they move between cameras, we first need to detect and track them locally - that is within the view of a single camera. Our basic assumption here is that the camera and the background scene remain static. The passing objects can be detected by finding pixels that deviate from the probabilistic model of the background scene. The static background assumption is fairly common, and there exist a variety of methods for object detection and local tracking. However, many of these methods consider individual pixels independent of each other, and thus often provide incomplete or rough object shapes. Part of our effort goes into modelling complete human silhouettes and detecting those instead of individual pixels.

When an object leaves the view of one camera, and after some time, enters the view of another camera, we want to associate the two local trajectories with a single global trajectory. Each local trajectory consists of multiple frames, therefore we compress it into a set of features that describe appearance of the object (like color distribution) and spatio-temporal features (like direction, or entry/leave timestamps). In principle, the feature-based association is a common problem in multi-object tracking. However, the standard methods are not immediately applicable as the motion of object is not smooth (because of the gap between fields of view). We have developed an online algorithm based on Bayesian inference in Infinite Gaussian Mixture Models (also known as Dirichlet Process Mixture Models). Read more.

Bayesian framework

The techniques that we develop follow from a common underlying principle - the Bayes theorem. The Bayesian framework embeds the relation between the unknown quantities - the states (e.g object identities) and observations - the data into a probability distribution p(states, data). Further, the distribution is decoupled into conceptually simpler parts: a model of internal state dynamics p(states) and a sensor model p(data|states). Given the measurements we can compute (often only approximately) posterior distributions of states. This distributions represent our belief about the states (e.g. about the identities of objects) following from the measurements. Such a framework places the designer's emphasis on accurate modelling of states and sensor noise, while computing posterior state densities is left to Bayesian numerical inference methods.

People

• Wojtek Zajdel: ideas, implementations, paper writing
• Ben Krose: (super-)vision, money, ideas, public relations
• A. Taylan Cemgil: ideas, implementations, paper writing
• Zoran Zivkovic: ideas, background segmentation, local tracking

Sponsors and Users

This project is founded by the Dutch Technology Foundation STW and the Dutch ministry of Economic Affairs, under grant number ANN.5312.

We also collaborate with companies: Eagle Vision, BV and TNO.