This paper presents a smart surveillance system named CASSANDRA, aimed at detecting instances of aggressive human behavior in public environments. A distinguishing aspect of CASSANDRA is the exploitation of complementary audio and video cues to disambiguate scene activity in real-life environments. From the video side, the system uses overlapping cameras to track persons in 3D and to extract features regarding the limb motion relative to the torso. From the audio side, it classifies instances of speech, screaming, singing, and kicking-object. The audio and video cues are fused with contextual cues (interaction, auxiliary objects); a Dynamic Bayesian Network (DBN) produces an estimate of the ambient aggression level. Our prototype system is validated on a realistic set of scenarios performed by professional actors at an actual train station to ensure a realistic audio and video noise setting.

This paper presents a multi-camera system to track multiple persons in complex, dynamic environments. Position measurements are obtained by carving out the space defined by foreground regions in the overlapping camera views and projecting these onto blobs on the ground plane. Person appearance is described in terms of the colour histograms in the various camera views of three vertical body regions (head-shoulder, torso, legs). The assignment of measurements to tracks (modelled by Kalman filters) is done in a non-greedy, global fashion based on ground plane position and colour appearance. The advantage of the proposed approach is that the decision on correspondences across cameras is delayed until it can be performed at the object-level, where it is more robust. We demonstrate the effectiveness of the proposed approach using data from three cameras overlooking a complex outdoor setting (train platform), containing a significant amount of lighting and background changes.

The capability to follow a person in a domestic environment is an important prerequisite for a robot companion. In this paper, a tracking algorithm is presented that makes it possible to follow a person using a small robot. This algorithm can track a person while moving around, regardless of the sometimes erratic movements of the legged robot. Robust performance is obtained by fusion of two algorithms, one based on salient features and one on color histograms. Re-initializing object histograms enables the system to track a person even when the illumination in the environment changes. By being able to re-initialize the system on run time using background subtraction, the system gains an extra level of robustness.