This paper is about an approach for tracking human targets through occlusions by applying a human model made of discrete parts.

This paper describes a novel approach to updating a template in a template tracking scheme. Specifically, the technique involves thermal imagery and makes appropriate use of domain knowledge to set thresholds based on physiological temperature constraints.

This paper presents a very good approach for tracking through occlusion by applying a probabilistic model to tracking features on an object. The authors present that the method works well even through partial occlusions.

This paper presents a way to detect the presence and number of people in an "interlock" form of a security portal.

This paper presents an end-to-end video system for detection, tracking, and classifying objects within the scope of a soccer match.

Video analytics for security and surveillance applications is becoming commonplace. Advances in algorithm robustness and low-cost video platforms have allowed analytics to become an ingredient for many different devices ranging from cameras to encoders to routers to storage. As algorithms become more refined, the analytics paradigm shifts from a human-support model to an automation model. In this context, ObjectVideo, the leader in intelligent video, has created a new concept in video analytics devices - the intelligent vision sensor (IVS). This device consists of a video imager and lens combined with an onboard processor and communication channel. This low-cost device turns video imagery into actionable information that can be used in building automation and business intelligence applications. This paper describes the technical and market drivers that facilitate the creation and adoption of the IVS device as well as a specific case study involving an application for heating, ventilation, and air conditioning (HVAC) and lighting control.

Loss prevention is a significant challenge in retail enterprises. A significant percentage of this loss occurs at point of sale (POS) terminals. POS data mining tools known collectively as exception based reporting (EBR) are helping retailers, but they have limitations as they can only work statistically on trends and anomalies in digital POS data. By applying video analytics techniques to POS transactions, it is possible to detect fraudulent or anomalous activity at the level of individual transactions. Very specific fraudulent behaviors that cannot be detected via POS data alone become clear when combined with video-derived data. ObjectVideo, a provider of intelligent video software, has produced a system called RetailWatch that combines POS information with video data to create a unique loss prevention tool. This paper describes the system architecture, algorithmic approach, and capabilities of the system, together with a customer case-study illustrating the results and effectiveness of the system.

Detecting stationary objects, such as an abandoned baggage or a parked vehicle is crucial in a wide range of video surveillance and monitoring applications. ObjectVideo, the leader in intelligent video software has been deploying commercial products to address these problems for the last 5 years. The ObjectVideo VEW and OnBoard system addresses these problems using an array of algorithms optimized for various scenario types and can be selected dynamically. This paper describes the key challenges and algorithms, and presents results on the standard i-LIDS dataset.

This paper describes an algorithm called ComMode (competitive mode estimation) for building a statistical background model of a complete video shot in the presence of independent camera motion, and independently moving foreground objects. Furthermore, as a by-product of the algorithm, the foreground is automatically segmented as a first step towards detection and tracking of video objects. The algorithm works by treating the intensity value at each pixel location as a time series, and segmenting it into statistically coherent modes. Initially, the most likely background mode is selected by a heuristic. Then, pixels competitively propagate their background mode to neighbouring pixels. The result is that each pixel location ends up with the most likely estimate for its background statistical model.

The Video Surveillance and Monitoring (VSAM) team at Carnegie Mellon University (CMU) has developed an end-to-end, multicamera surveillance system that allows a single human operator to monitor activities in a cluttered environment using a distributed network of active video sensors. Video understanding algorithms have been developed to automatically detect people and vehicles, seamlessly track them using a network of cooperating active sensors, determine their three-dimensional locations with respect to a geospatial site model, and present this information to a human operator who controls the system through a graphical user interface. The goal is to automatically collect and disseminate real-time information to improve the situational awareness of security providers and decision makers. The feasibility of real-time video surveillance has been demonstrated within a multicamera testbed system developed on the campus of CMU. This paper presents an overview of the issues and algorithms involved in creating this semiautonomous, multicamera surveillance system.

UTOMATED video surveillance addresses real-time observation of people and vehicles within a busy environment, leading to a description of their actions and interactions. The technical issues include moving object detection and tracking, object classification, human motion analysis, and activity understanding, touching on many of the core topics of computer vision, pattern analysis, and aritificial intelligence. Video surveillance has spawned large research projects in the United States, Europe, and Japan, and has been the topic of several international conferences and workshops in recent years. There are immediate needs for automated surveillance systems in commercial, law enforcement, and military applications. Mounting video cameras is cheap, but finding available human resources to observe the output is expensive. Although surveillance cameras are already prevalent in banks, stores, and parking lots, video data currently is used only “after the fact” as a forensic tool, thus losing its primary benefit as an active, real-time medium. What is needed is continuous 24-hour monitoring of surveillance video to alert security officers to a burglary in progress or to a suspicious individual loitering in the parking lot, while there is still time to prevent the crime. In addition to the obvious security applications, video surveillance technology has been proposed to measure traffic flow, detect accidents on highways, monitor pedestrian congestion in public spaces, compile consumer demographics in shopping malls and amusement parks, log routine maintainence tasks at nuclear facilities, and count endangered species. The numerous military applications include patrolling national borders, measuring the flow of refugees in troubled areas, monitoring peace treaties, and providing secure perimeters around bases and embassies. The 11 papers in this special section illustrate topics and techniques at the forefront of video surveillance research. These papers can be loosely organized into three categories. Detection and tracking involves real-time extraction of moving objects from video and continuous tracking over time to form persistent object trajectories. C. Stauffer and W.E.L. Grimson introduce unsupervised statistical learning techniques to cluster object trajectories produced by adaptive background subtraction into descriptions of normal scene activity. Viewpoint-specific trajectory descriptions from multiple cameras are combined into a common scene coordinate system using a calibration technique described by L. Lee, R. Romano, and G. Stein, who automatically determine the relative exterior orientation of overlapping camera views by observing a sparse set of moving objects on flat terrain. Two papers address the accumulation of noisy motion evidence over time. R. Pless, T. Brodský, and Y. Aloimonos detect and track small objects in aerial video sequences by first compensating for the self-motion of the aircraft, then accumulating residual normal flow to acquire evidence of independent object motion. L. Wixson notes that motion in the image does not always signify purposeful travel by an independently moving object (examples of such “motion clutter” are wind-blown tree branches and sun reflections off rippling water) and devises a flow-based salience measure to highlight objects that tend to move in a consistent direction over time. Human motion analysis is concerned with detecting periodic motion signifying a human gait and acquiring descriptions of human body pose over time. R. Cutler and L.S. Davis plot an object’s self-similarity across all pairs of frames to form distinctive patterns that classify bipedal, quadripedal, and rigid object motion. Y. Ricquebourg and P. Bouthemy track apparent contours in XT slices of an XYT sequence volume to robustly delineate and track articulated human body structure. I. Haritaoglu, D. Harwood, and L.S. Davis present W4, a surveillance system specialized to the task of looking at people. The W4 system can locate people and segment their body parts, build simple appearance models for tracking, disambiguate between and separately track multiple individuals in a group, and detect carried objects such as boxes and backpacks. Activity analysis deals with parsing temporal sequences of object observations to produce high-level descriptions of agent actions and multiagent interactions. In our opinion, this will be the most important area of future research in video surveillance. N.M. Oliver, B. Rosario, and A.P. Pentland introduce Coupled Hidden Markov Models (CHMMs) to detect and classify interactions consisting of two interleaved agent action streams and present a training method based on synthetic agents to address the problem of parameter estimation from limited real-world training examples. M. Brand and V. Kettnaker present an entropyminimization approach to estimating HMM topology and

In this paper a process is described for analysing the motion of a human target in a video stream. Moving targets are detected and their boundaries extracted. From these, a "star" skeleton is produced. Two motion cues are determined from this skeletonization: body posture, and cyclic motion of skeleton segments. These cues are used to determine human activities such as walking or running, and even potentially, the target's gait. Unlike other methods, this does not require an a priori human model, or a large number of "pixels on target". Furthermore, it is computationally inexpensive, and thus ideal for real-world video applications such as outdoor video surveillance.

This paper describes an end-to-end method for extracting moving targets from a real-time video stream, classifying them into predefined categories according to image-based properties, and then robustly tracking them. Moving targets are detected using the pixel wise difference between consecutive image frames. A classification metric is applied these targets with a temporal consistency constraint to classify them into three categories: human, vehicle or background clutter. Once classified targets are tracked by a combination of temporal differencing and template matching. The resulting system robustly identifies targets of interest, rejects background clutter and continually tracks over large distances and periods of time despite occlusions, appearance changes and cessation of target motion.