Diffusion models indirectly estimate the probability density over a data space, which can be used to study its structure. In this work, we show that geodesics can be computed in diffusion latent space, where the norm induced by the spatially-varying inner product is inversely proportional to the probability density. In this formulation, a path that traverses a high density (that is, probable) region of image latent space is shorter than the equivalent path through a low density region. We present algorithms for solving the associated initial and boundary value problems and show how to compute the probability density along the path and the geodesic distance between two points. Using these techniques, we analyze how closely video clips approximate geodesics in a pre-trained image diffusion space. Finally, we demonstrate how these techniques can be applied to training-free image sequence interpolation and extrapolation, given a pre-trained image diffusion model.

In this work, we formulate a novel framework for adversarial robustness using the manifold hypothesis. This framework provides sufficient conditions for defending against adversarial examples. We develop an adversarial purification method with this framework. Our method combines manifold learning with variational inference to provide adversarial robustness without the need for expensive adversarial training. Experimentally, our approach can provide adversarial robustness even if attackers are aware of the existence of the defense. In addition, our method can also serve as a test-time defense mechanism for variational autoencoders.

Deep learning approaches to natural language processing have made great strides in recent years. While transformer models have demonstrated impressive knowledge and reasoning capabilities, it is unclear how produced symbols are grounded in data from the world. In this paper, we explore the development of a private language for visual data representation by training emergent language (EL) encoders/decoders in both i) a traditional referential game environment and ii) a contrastive learning environment utilizing a within-class matching training paradigm. An additional classification layer-utilizing neural machine translation and random forest classification-was used to transform symbolic representations (sequences of integer symbols) to class labels. These methods were applied in two experiments focusing on object recognition and action recognition. For object recognition, a set of sketches produced by human participants from real imagery was used and for action recognition, 2D trajectory images were generated from 3D motion capture systems. In order to interpret the symbols produced for data in each experiment, a Gradient-weighted Class Activation Mapping (GradCAM) method was used to identify pixel regions indicating semantic features which contribute evidence towards symbols in learned languages. Results indicate that: i) symbols used to represent images appear to shift focus between different semantic components in an image and ii) this shift occurs gradually over the course of a sentence.

We present a diffusion-based image morphing approach with perceptually-uniform sampling (IMPUS) that produces smooth, direct and realistic interpolations given an image pair. The embeddings of two images may lie on distinct conditioned distributions of a latent diffusion model, especially when they have significant semantic difference. To bridge this gap, we interpolate in the locally linear and continuous text embedding space and Gaussian latent space. We first optimize the endpoint text embeddings and then map the images to the latent space using a probability flow ODE. Unlike existing work that takes an indirect morphing path, we show that the model adaptation yields a direct path and suppresses ghosting artifacts in the interpolated images. To achieve this, we propose a heuristic bottleneck constraint based on a novel relative perceptual path diversity score that automatically controls the bottleneck size and balances the diversity along the path with its directness. We also propose a perceptually-uniform sampling technique that enables visually smooth changes between the interpolated images. Extensive experiments validate that our IMPUS can achieve smooth, direct, and realistic image morphing and is adaptable to several other generative tasks.

We introduce Symbolic Variational Autoencoders which generate images from symbols that represent semantic concepts. Unlike generic Variational Autoencoders (VAEs) or Generative Adversarial Networks (GANs), the latent distribution from the Symbolic Variational Autoencoder is discrete. The symbols are learned in a completely unsupervised manner by reconstructing images from symbolic encodings. We demonstrate the efficacy of our symbolic approach on the MNIST and FashionMNIST datasets. Results indicate that symbolic encodings naturally form a grammar, where unique strings of symbols map to different semantic concepts. We further explore how changing these symbols affects the final image that is generated.

We present a video analytics system combining railcar detection, classification, Federal Railroad Admin. (FRA) text identification, and logo detection into a system for locomotive transportation and yard management. Existing RFID-based systems are limited by sensor deployment and cannot visually identify railcars when they are away. As there are typically tens of tracks and hundreds of railcars in a yard, an automatic vision system is desirable. The proposed AI system is developed for autonomous yard inventory checking, such that the arrival, departure, and movement of individual railcars can be automatically monitored and managed in the facility. Our system consists of multiple cameras with edge computing devices installed at check points (track entrances and branches), such that visual detection and tracking of railcars can be performed and meta-data can be exchanged. After knowing the railcar locations and types, scene text detection is performed to search and recognize FRA ID markings and logos that can uniquely identify each railcar. Information fusion a database in the central hub can further improve railcar identification and reduce errors. Early results on real-world field collected data demonstrate the efficacy of the proposed approach.

An interactive advertising system is proposed comprising of a flat screen display, a collocated sensing camera and an outcome feedback camera. The display presents an Avatar that has the goal of convincing people to enter a store. The Avatar has a repertoire of actions that it can perform such as offer a coupon or present a piece of merchandise. A sensing camera is used to represent the state of a possible customer. State variables include affective pose, facial expressions, gaze direction and eyeball movement. The outcome feedback camera is used to detect if and when a customer elects to enter the store. Given access to such feedback, a reinforcement learning algorithm is then used to construct a policy that indicates which actions should be taken as a function of observed customer state. The efficacy of this approach is considered via synthetic experiments as well as real world demonstrations. The paper includes a discussion regarding new descriptors based on emergent languages that can be used to augment the state space representation.

We present a multimodal sensor system for wound assessment and pressure ulcer care. Multiple imaging modalities including RGB, three- dimensional (3-D) depth, thermal, multispectral, and chemical sensing are integrated into a portable hand-held probe for real-time wound assessment. Analytic and quantitative algorithms for various assessments including tissue composition, wound measurement in 3-D, temperature profiling, spectral, and chemical vapor analysis are developed. After each assessment scan, 3-D models of the wound are generated on the fly for geometric measurement, while multimodal observations are analyzed to estimate healing progress. Collaboration between developers and clinical practitioners was conducted at the Charlie Norwood VA Medical Center for in-field data collection and experimental evaluation. A total of 133 assessment sessions from 23 enrolled subjects were collected, on which the multimodal data were analyzed and validated with respect to clinical notes associated with each subject. The system can be operated by nontechnical caregivers on a regular basis to aid wound assessment and care. A web portal front-end was developed for clinical decision and telehealth support, where all historical patient data including wound measurements and analysis can be organized online.

We present a real-time depth based computer vision system for pressure ulcer prevention, in-bed patient care and monitoring. Our system can effectively determine whether or not a mobility-compromised patient has been correctly repo-sitioned at the required frequency. A depth sensor is used to detect and recognize patient movements, motion patterns, and pose positions. If the patient has stayed in an unchanged pose for too long and needs pressure releasing movements, our system can notify caregivers for repositioning or assistance. Privacy concerns are mitigated by removing the RGB components of the video stream from the camera capturing, and only processing depth measurements. We collaborated with clinical practitioners at the Charlie Norwood VA Medical Center for in-field data collection and experimental evaluation. A web portal front-end is developed such that all historical patient movements, pose positions, and repositioning data can be organized to support telehealth applications.

We propose a video analytic system to segment and track clouds for the purpose of solar irradiance prediction. Ground-based imaging sensors are used to monitor potential sun occlusions for solar irradiance drop prediction, which can be used to assist power ramp control strategies. Sky images are first rectified to remove fisheye artifacts. Cloud pixels are segmented and classified into low-to-high transparencies. Evolution of cloud boundaries are tracked using optical flow. Tracking extrapolation predicts future cloud movements and deformations as well as potential sun occlusions. To accurately estimate solar irradiance drop, we propose a novel scheme based on back-projecting the predicted sun occlusion onto the evolving cloud boundary to count for cloud transparency and irradiance drop estimation. Experimental results show that short-term solar irradiance drop is predictable with reasonable accuracy.

We propose an efficient framework for large-scale 3D reconstruction from a large set of photos following the Structure-from-Motion (SfM) paradigm with divide-conquer and fusion. Our main novelty is to ensure commonality from overlaps between image sets corresponding to their reconstructions, which facilitates effective stitching and fusion. Specifically, such commonality is ensured by selecting a set of duplicated images (which are termed anchor images) in adjacent image sets prior to the 3D reconstruction. The anchor images can assist accurate fusion of the 3D point clouds. We describe an efficient RANSAC scheme for pairwise stitching. Our method is intuitively scalable to large site reconstruction via subdivision and fusion following a graph construct. We further describe another RANSAC algorithm to improve loop closure in our anchor image approach. Experimental results on reconstructing a large portion of a university campus demonstrate the efficacy of our method.

We investigate the use of a vision-based system capable of estimating social states such as rapport and hostility. We study the correlation between interpretations automatically generated by our system and those reported by social scientists. Our multi-camera vision system collects visual cues including location (proximity), motion, pose, gaze, and facial expressions in real-time from multiple subjects moving freely in an unconstrained environment. We performed experiments on 80+ subjects. Preliminary regression analysis suggests high correlation between machine distilled time series signals and assessments made by human experts.

We present a video event visualization system that can render steerable 3D views of tracked targets onto a reconstructed 3D site representation. The framework takes object tracking meta-data generated from a multi-camera event tracking system as input and produces an immersive 3D playback as a representation of the observation. This 3D representation can provide users seeing as it happens of the events for surveillance applications. Our system can further “animate” the virtual viewing camera and generate a first-person immersive playback of the event, either from the trajectory of a specified real-world target or from a virtual avatar. Such synthetic view can provide additional insights for event recognition for on-line monitoring, investigation, and forensic applications.

A reinforcement learning model, which allows for an agent to interact with a simulated 3D learning environment under the initial guidance of an all knowing oracle is proposed. Methods are presented that allow the agent to learn how to perform a set of task oriented actions. It is then hypothesized that the ability to recognize an action may in fact be a byproduct of learning how to perform an action. Evidence supporting this conjecture is presented using both simulated and real world imagery.

A key assumption of traditional machine learning is that both the training and test data share the same distribution. However, this assumption does not hold in many real-world scenarios. For example, in facial expression recognition, the appearance of an expression may vary significantly for different people. Previous work has shown that learning from adequate person-specific data can improve facial expression recognition results. However, because of the difficulties of data collection and labeling, person-specific data is usually very sparse in real-world applications. Learning from the sparse data may suffer from serious over-fitting. In this paper, we propose to learn a person-specific facial expression model through transfer learning. By transferring the informative knowledge from other people, it allows us to learn an accurate person-specific model for a new subject with only a small amount of his/her specific data.

Given the size and congestion associated with a standard convenience store, it is possible to achieve automated site-wide tracking for each customer via a network of in-store video cameras. For the purposes of reducing customer related shrinkage (theft), it may be advantageous to produce summary videos of the activities associated with each customer. As the customer approaches an exit or checkout station, the cashier could quickly review the associated summery video for the purposes of determining whether or not a theft has taken place. Such a summery video would be automatically generated from clips produced by a network of in-store cameras. The value of the summery videos can be determined based on their brevity and the cashier's ability to detect the presence or absence of a theft event. To this end we propose that a site-wide tracking system coupled with a clip nomination process that is based on a set of action saliency measures be used to produce the desired summery videos.

Effective perimeter protection mechanisms for industrial sites and critical infrastructure must contend with a large variety of potential threats as well as with the fact that normal site activity can be both complex and diverse. This paper documents the development of a system level approach capable of functioning under such challenging conditions. A multi-view tracking system is used to provide real-time site wide trajectories of all observed individuals. A Radar-based system is also used for tracking if and when camera coverage of various regions is not available. Track information is then analyzed with respect to articulated motion analysis, complex event analysis and normalcy analysis. In addition, object recognition is used to classify left behind objects using high resolution PTZ imagery. A real-time integrated version of this comprehensive approach to perimeter protection was deployed using a single standard off-the-shelf desktop computer.

Summary form only given. The rapid evolution of tools and software systems to design experiments, automatically monitor, collect and warehouse large amounts of data, from applications such as life sciences and industrial processes has resulted in a new paradigm shift. This change of paradigm is so fast that some of the practices for optimization and management of these processes that were valid only 5–10 years ago may no longer be fully acceptable or sufficient for today's business optimization and management. This has a direct influence on the best practices for knowledge discovery and management of the discovered knowledge in real-world data mining applications. Establishing and managing a real-world data mining project in any domain, in particular in today's life science industry, is not a trivial task. A few approaches have been proposed in the literature. However, initiation and successful management of such efforts may depend on where a given case study fits in the overall classification of data mining approaches. Today's knowledge discovery from data can be classified in several ways: (i) data mining on engineered systems (e.g. complex equipment) or systems designed by nature (e.g. life sciences), (ii) explanatory or predictive data mining, (iii) data mining from static data (e.g. data warehouse) or dynamic data (e.g. data streams), (iv) user operated or automated data mining. There could still be other ways to classify data mining applications. This talk provides an overview of the above listed knowledge discovery applications. We provide examples where we demonstrate how small or large amounts of data, when understood from a real-world data mining point of view and the required data is properly integrated, can result in novel knowledge discovery case studies. We explain motivations and challenges of establishing real-world dat

In this paper, we address the problem of online learning to recognize people from visual appearances, a prerequisite step towards building a fully intelligent and context-aware smart environment. While the trajectories of tracked individuals are responsible for producing samples to the appearance signature learning process, it is highly risky to directly label these appearance samples with tracker IDs, due to possible tracker switches and temporary tracker losses. Through the exploration of trajectory fidelity in terms of temporal continuity and spatial locality, we show that the side information from tracking, in the form of pairwise constraints, such as “must-link” and “cannot-link”, could significantly benefit signature learning. Furthermore, to learn and update an online identity signature pool, a two-step approach is proposed: 1) a data clustering step based on spectral kernel learning with pairwise constraints, and 2) a large-margin based discriminative signature model learning step. A real-world setup in a smart office environment is used to evaluate the performance of the learning paradigm. Consistent recognition of individuals from live videos verifies the efficacy and effectiveness of our proposal.

We propose a shape model fitting algorithm that uses linear programming optimization. Most shape model fitting approaches (such as ASM, AAM) are based on gradient-descent-like local search optimization and usually suffer from local minima. In contrast, linear programming (LP) techniques achieve globally optimal solution for linear problems. In [1], a linear programming scheme based on successive convexification was proposed for matching static object shape in images among cluttered background and achieved very good performance. In this paper, we rigorously derive the linear formulation of the shape model fitting problem in the LP scheme and propose an LP shape model fitting algorithm (LPSM). In the experiments, we compared the performance of our LPSM with the LP graph matching algorithm(LPGM), ASM, and a CONDENSATION based ASM algorithm on a test set of PUT database. The experiments show that LPSM can achieve higher shape fitting accuracy. We also evaluated its performance on the fitting of some real world face images collected from internet. The results show that LPSM can handle various appearance outliers and can avoid local minima problem very well, as the fitting is carried out by LP optimization with l1norm robust cost function.

In this work we propose a dynamic scene model to provide information about the presence of salient motion in the scene, and that could be used for focusing the attention of a pan/tilt/zoom camera, or for background modeling purposes. Rather than proposing a set of saliency detectors, we define what we mean by salient motion, and propose a precise model for it. Detecting salient motion becomes equivalent to detecting a model change. We derive optimal online procedures to solve this problem, which enable a very fast implementation. Promising results show that our model can effectively detect salient motion even in severely cluttered scenes, and while a camera is panning and tilting.

Intelligent video in urban settings can be challenging due the presence of crowds, clutter, poor camera placement and continuously changing light conditions. The surveillance of sports venues is particularly difficult, because thousands of people can enter or exit a venue in short periods of time. This paper presents a case study of successfully monitoring a sports venue using a multi-camera multi-target tracking system. The system performed site-wide tracking throughout a network of calibrated cameras and was able to accurately track thousands of people in real-time under challenging conditions. The extracted tracking information was used to detect a range of real-time events such as crowd formation, left luggage, and loitering. In addition all video,track and event information was indexed and stored to allow operators to perform playback and forensic search. This paper will present an overview of the deployed system and discuss the challenges that were encountered during the deployment.

Landmark labeling of training images is essential for many learning tasks in computer vision, such as object detection, tracking, and alignment. Image labeling is typically conducted manually, which is both labor-intensive and error-prone. To improve this process, this paper proposes a new approach to estimate a set of landmarks for a large image ensemble with only a small number of manually labeled images from the ensemble. Our approach, named semi-supervised least-squares congealing, aims to minimize an objective function defined on both labeled and unlabeled images. A shape model is learnt on-line to constrain the landmark configuration. We also employ a partitioning strategy to allow coarse-to-fine landmark estimation. Extensive experiments on facial images show that our approach can reliably and accurately label landmarks for a large image ensemble starting from a small number of manually labeled images, under various challenging scenarios.

We propose a subspace learning algorithm for face recognition by directly optimizing recognition performance scores. Our approach is motivated by the following observations: 1) Different face recognition tasks (i.e., face identification and verification) have different performance metrics, which implies that there exist distinguished subspaces that optimize these scores, respectively. Most prior work focused on optimizing various discriminative or locality criteria and neglect such distinctions. 2) As the gallery (target) and the probe (query) data are collected in different settings in many real-world applications, there could exist consistent appearance incoherences between the gallery and the probe data for the same subject. Knowledge regarding these incoherences could be used to guide the algorithm design, resulting in performance gain. Prior efforts have not focused on these facts. In this paper, we rigorously formulate performance scores for both the face identification and the face verification tasks, provide a theoretical analysis on how the optimal subspaces for the two tasks are related, and derive gradient descent algorithms for optimizing these subspaces. Our extensive experiments on a number of public databases and a real-world face database demonstrate that our algorithm can improve the performance of given subspace based face recognition algorithms targeted at a specific face recognition task.

This paper studies image alignment, the problem of learning a shape and appearance model from labeled data and efficiently fitting the model to a non-rigid object with large variations. Given a set of images with manually labeled landmarks, our model representation consists of a shape component represented by a point distribution model and an appearance component represented by a collection of local features, trained discriminatively as a two-class classifier using boosting. Images with ground truth landmarks are the positive training samples while those with perturbed landmarks are considered as negatives. Enabled by piece-wise affine warping, corresponding local feature positions across all training samples form a hypothesis space for boosting. Image alignment is performed by maximizing the boosted classifier score, which is our distance measure, through iteratively mapping the feature positions to the image, and computing the gradient direction of the score with respect to the shape parameter. We apply this approach to human body alignment from surveillance-type images. We conduct experiments on the MIT pedestrian database where the body size is approximately 110 times 46 pixels, and demonstrate our real-time alignment capability.

This paper presents a novel distributed framework for multi-target tracking with an efficient data association computation. A decentralized representation of trackers’ motion and association variables is adopted. Considering the interleaved nature of data association and tracker filtering, the multi-target tracking is formulated as a missing data problem, and the solution is found by the proposed variational EM algorithm. We analytically show that 1) the posteriori distributions of trackers’ motions (the real interests in terms of tracking applications) can be effectively computed in the E-step of the EM iterations, and 2) the solution of trackers’ association variables can be pursued under a derived graph-based discrete optimization formulation, thus efficiently estimated in the M-step by the recently emerging graph optimization algorithms. The proposed approach is very general such that sophisticated data association priori and likelihood function can be easily incorporated. This general framework is tested with both simulation data and real world surveillance video. The reported qualitative and quantitative studies verify the effectiveness and low computational cost of the algorithm.

Computer vision approaches for retail applications can provide value far beyond the common domain of loss prevention. Gaining insight into the movement and behaviors of shoppers is of high interest for marketing, merchandizing, store operations and data mining. Of particular interest is the process of purchase decision making. What catches a customers attention? What products go unnoticed? What does a customer look at before making a final decision? Towards this goal we presents a system that detects and tracks both the location and gaze of shoppers in retail environments. While networks of standard overhead store cameras are used for tracking the location of customers, small in-shelf cameras are used for estimating customer gaze. The presented system operates robustly in real-time and can be deployed in a variety of retail applications.

The ability to determine the identity of a skull found at a crime scene is of critical importance to the law enforcement community. Traditional clay-based methods attempt to reconstruct the face so as to enable identification of the deceased by members of the general public. However, these reconstructions lack consistency from practitioner to practitioner and it has been shown that the human recognition of these reconstructions against a photo gallery of potential victims is little better than chance. In this talk we discuss the automation of the reconstruction process. For a given skull, a data-driven 3D generative model of the face is constructed using a database of CT head scans. The reconstruction can be constrained based on prior knowledge such as age and or weight. To determine whether or not these reconstructions have merit, geometric methods for comparing reconstructions against a gallery of facial images are proposed. First, active appearance models are used to automatically detect a set of facial landmarks on each image. These landmarks are associated with 3D points on the reconstruction. Direct comparison of the reconstruction is problematic since in general the camera geometry used for image capture is unknown and there are uncertainties associated with the reconstruction and landmark detection processes. The first method of comparison uses constrained optimization to determine the optimal projection of the reconstruction on to the image. Residuals are then analyzed resulting in a ranking of the gallery. The second method uses boosting to learn which points are both reliable and discriminating. This results in a match/no-match classifier. Experimental evidence indicating that skull recognition from facial images can be achieved is presented.

Using a detect and track paradigm, we present a surveillance framework where each camera uses local resources to perform real-time person detection. These detections are then processed by a distributed site-wide tracking system. The detectors themselves are based on boosted user-defined exemplars, which capture both appearance and shape information. The detectors take integral images of both intensity and Sobel responses as input. This data representation enables efficient processing without relying on background subtraction or other motion cues. View-specific person detectors are constructed by iteratively presenting the boosting algorithm with training data associated with each individual camera. These detections are then transmitted from a distributed set of tracking clients to a server, which maintains a set of site-wide target tracks. Automatic calibration methods allow for tracking to be performed in a ground plane representation, which enables effective camera hand-off. Factors such as network latencies and scalability will be discussed.

Active Appearance Models (AAMs) represent the shape and appearance of an object via two low-dimensional subspaces, one for shape and one for appearance. AAMs for facial images are currently receiving considerable attention from the computer vision community. However, most existing work focuses on fitting AAMs to a single image. For many applications, effectively fitting an AAM to video sequences is of critical importance and challenging, especially considering the varying quality of real-world video content. This paper proposes a hybrid model to address this problem. Both a generic AAM and a subject-specific model are employed simultaneously in the proposed fitting scheme. Experimental results from outdoor surveillance video sequences demonstrate the improved image registration across video frames and faster fitting convergence. 1

The ability to determine the identity of a skull found at a crime scene is of critical importance to the law enforcement community. Traditional clay-based methods attempt to reconstruct the face so as to enable identification of the deceased by members of the general public. However, these reconstructions lack consistency from practitioner to practitioner and it has been shown that the human recognition of these reconstructions against a photo gallery of potential victims is little better than chance. In this paper we propose the automation of the reconstruction process. For a given skull, a data-driven 3D generative model of the face is constructed using a database of CT head scans. The reconstruction can be constrained based on prior knowledge such as age and or weight. To determine whether or not these reconstructions have merit, geometric methods for comparing reconstructions against a gallery of facial images are proposed. First, active shape models are used to automatically detect a set of facial landmarks on each image. These landmarks are associated with 3D points on the reconstruction. Direct comparison of the reconstruction is problematic since in general the camera geometry used for image capture is unknown and there are uncertainties associated with the reconstruction and landmark detection processes. The first method of comparison uses constrained optimization to determine the optimal projection of the reconstruction on to the image. Residuals are then analyzed resulting in a ranking of the gallery. The second method uses boosting to learn which points are both reliable and discriminating. This results in a match/no-match classifier. Experimental evidence indicating that skull recognition from facial images can be achieved is presented.

In this work we develop appearance models for computing the similarity between image regions containing deformable objects of a given class in realtime. We introduce the concept of shape and appearance context. The main idea is to model the spatial distribution of the appearance relative to each of the object parts. Estimating the model entails computing occurrence matrices. We introduce a generalization of the integral image and integral histogram frameworks, and prove that it can be used to dramatically speed up occurrence computation. We demonstrate the ability of this framework to recognize an individual walking across a network of cameras. Finally, we show that the proposed approach outperforms several other methods.

Active Appearance Models (AAMs) represent the shape and appearance of an object via two low-dimensional subspaces, one for shape and one for appearance. AAMs for facial images are currently receiving considerable attention from the vision community. However, most existing work focuses on fitting AAMs to high-quality facial images. For many applications, effectively fitting an AAM to low-resolution facial images is of critical importance. This paper addresses this challenge from two aspects. On the modeling side, we propose an iterative AAM enhancement scheme, which not only results in increased fitting speed, but also improves the fitting robustness. For fitting AAMs to low-resolution images, we build a multi-resolution AAM and show that the best fitting performance is obtained when the model resolution is slightly higher than the facial image resolution. Experimental results using both indoor video and outdoor surveillance video are presented. 1

A number of surveillance scenarios require the detection and tracking of people. Although person detection and counting systems are commercially available today, there is need for further research to address the challenges of real world scenarios. The focus of this work is the segmentation of groups of people into individuals. One relevant application of this algorithm is people counting. Experiments document that the presented approach leads to robust people counts.

The main focus of this work is the integration of feature grouping and model based segmentation into one consistent framework. The algorithm is based on partitioning a given set of image features using a likelihood function that is parameterized on the shape and location of potential individuals in the scene. Using a variant of the EM formulation, maximum likelihood estimates of both the model parameters and the grouping are obtained simultaneously. The resulting algorithm performs global optimization and generates accurate results even when decisions can not be made using local context alone. An important feature of the algorithm is that the number of people in the scene is not modeled explicitly. As a result no prior knowledge or assumed distributions are required. The approach is shown to be robust with respect to partial occlusion, shadows, clutter, and can operate over a large range of challenging view angles including those that are parallel to the ground plane. Comparisons with existing crowd segmentation systems are made and the utility of coupling crowd segmentation with a temporal tracking system is demonstrated.

Indoor video surveillance networks as typically used in retail-networks need to be consistently calibrated with respect to a global coordinate system to provide the spatial context required for automatic recognition. This can be problematic if the fields of view of the cameras are disjoint. The paper presents a strategy to calibrate disjoint views into one coordinate system. The system is based on a device which can calculate its current position and orientation with respect to the global coordinate frame while it traverses the entire site without requiring direct line of sight between all sets of points. The paper explains why more traditional approaches have disadvantages and presents experimental results using the new method.

This paper proposes a method for three-dimensional reconstruction of surfaces that takes advantage of the symmetry resulting from alternating the positions of a camera and a light source. This set up allows for use of the Helmholtz reciprocity principle to recover the shape of smooth surfaces with arbitrary bidirectional reflectance distribution functions without requiring the presence of texture, as well as for exploiting mutual occlusions between images. Different from previous methods, the technique works with as few as one reciprocal pair, and recovers surface depth and orientation simultaneously by finding the global minimum of an error function via dynamic programming. Since the error is a function not just of depth but of surface orientation as well, the reconstruction is subject to tighter geometric constraints. Given a current estimate of surface geometry and intensity measurements in one image, Helmholtz reciprocity is used to predict the pixel intensity values of the second image. The dynamic program finds the reconstruction that minimizes the total difference between the predicted and measured intensity values. This approach allows for the reconstruction of surfaces displaying specularities and regions of high curvature, which is a challenge commonly encountered in the optical inspection of industrial parts. Results with real data show the quality of the reconstruction obtained with the proposed algorithm.

Describes a method for extracting 3D events from a 3D volume of seismic data for the purposes of geophysical interpretation. An event is the recorded waveform caused by a seismic scatterer. It can be modeled as a hyperboloid with local perturbations. Extraction is difficult because of the local perturbations and, particularly, because events cross one another. The original data is in the form of 2D time sections which are vertical slices through the data volume. Once the event has been extracted, it is represented by fitting two types of surfaces. The first surface is a hyperboloid. The parameters that define the hyperboloid are used to determine geophysical parameters such as the position of the scatterer and the Earth's average propagation velocity. The second is a smooth approximating surface calculated using a regularising term. This surface is useful for visualising regions where the event pulls away from pure hyperbolic form.>

This paper describes a method for extracting 3D events from a 3D volume of seismic data for the purposes of geophysical interpretation. An event is the recoded waveform caused by a seismic scatterer. It can be modeled as hyperboloids with local perturbations. Extraction is difficult because of the local perturbations and, particularly, because events cross one another. The original data is in the form of 2D time sections which are vertical slices through the data volume. Constraints based on the physics of the seismic experiment are used to extract 2D event curves from the time sections and then combine them into 3D events. Two types of surfaces are fitted to the 3D event so that meaningful interpretations can be made.

eMail peter.tu*Q>eng.ox.ac.uk In this paper we introduce a three stage approach for extracting events from a seismic time section. The approach incorporates recent, techniques from computer vision such as deformable templates and multiple hypothesis tracking, and also takes advantage of constraints arising from the physics of seismic reflectors. First, a 2D local matched fdtering scheme is used to reduce the time section to a collection of event tokens suitable for tracking with a Kalman filter. Second, a multiple tracking system is used to analyse regions with crossing events. By using a dynamic model of event shape parameters, Kalman filters are able to track events that deviate from an ideal form. Finally, based on events found through Kalman filtering, flexible templates are used to exploit similarity between events. Due to the global nature of the flexible template search process, even events with sections of poor visibility can be identified. Based on this research, a semi-automatic system for extracting seismic events has been developed.