Representations in the form of Symmetric Positive Definite (SPD) matrices have been popularized in a variety of visual learning applications due to their demonstrated ability to capture rich second-order statistics of visual data. There exist several similarity measures for comparing SPD matrices with documented benefits. However, selecting an appropriate measure for a given problem remains a challenge and in most cases, is the result of a trial-and-error process. In this paper, we propose to learn similarity measures in a data-driven manner. To this end, we capitalize on the αβ-log-det divergence, which is a meta-divergence parametrized by scalars α and β, subsuming a wide family of popular information divergences on SPD matrices for distinct and discrete values of these parameters. Our key idea is to cast these parameters in a continuum and learn them from data. We systematically extend this idea to learn vector-valued parameters, thereby increasing the expressiveness of the underlying non-linear measure. We conjoin the divergence learning problem with several standard tasks in machine learning, including supervised discriminative dictionary learning and unsupervised SPD matrix clustering. We present Riemannian gradient descent schemes for optimizing our formulations efficiently, and show the usefulness of our method on eight standard computer vision tasks.

A major component of an efficient farming strategy is the precise detection and characterization of plant deficiencies followed by the proper deployment of fertilizers. Through the thoughtful utilization of modern computer vision techniques, it is possible to achieve positive financial and environmental results for these tasks. This work introduces an automation framework that attempts to address the three main drawbacks of existing approaches: 1) lack of generality (methods are tuned for specific data sets); 2) difficulty to apply in variable field conditions; and 3) lack of tool sophistication that limits their applicability. The cultivation of corn lies in the core of the American and global economy with 81.7 million acres harvested only in the USA for the year 2018. The ubiquity of its cultivation makes it an ideal candidate to highlight the large economic benefits from even a small improvement in nutrient deficiency detection. The proposed methodology utilizes drone collected images to detect nitrogen (N) deficiencies in maize fields and assess their severity using low-cost RGB sensors. The proposed methodology is twofold. A low complexity recommendation scheme identifies candidate plants exhibiting N deficiency and, with minimal interaction, assists the annotator in the creation of a training data set that is then used to train an object detection deep neural network. Results on data from experimental fields support the merits of the proposed methodology with mean average precision for the detection of N-deficient leaves reaching 82.3%.Note to Practitioners—The motivation behind this article is the problem of inefficient fertilizer application in corn fields throughout the cultivation season. Current widely spread techniques to counter plant malnutrition suggest the application of excessive amounts of nitrogen fertilizer prior to seeding or the uniform application during the plant growth. These practices result in financial losses and have severe environmental consequences, e.g., the dead zone in the Gulf of Mexico. We propose an automation framework that automatically detects corn nitrogen deficiencies in the field during the plants’ growth, and to achieve our goal, we employ low-cost robotic platforms and RGB sensors. The framework that we developed is able to detect the characteristic pattern of nitrogen deficiency on corn leaves and provide an estimation of the in-field spatial variability of the deficiency.

Financial and social elements of modern societies are closely connected to the cultivation of corn. Due to its massive production, deficiencies during the cultivation process directly translate to major financial losses. Since proper surveillance in a large scale is still very challenging, the companies that specialize in optimizing crop yield are trying to address the problem at its root by developing hybrid plants able to resist the harsh conditions of the field. The selection of the best hybrid is not easy and every year hundreds of test plants with different phenotypic characteristics are planted while their performance is quantified by inconsistent and rough measurements gathered by humans. We propose a pipeline that takes advantage of the structure from motion technology to create a detailed 3D point cloud of a few plants and segment it into the basic elements of the scene; the ground, the plants, the plant stems, and the plant leaves. The focus is on the segmentation process through which several phenotypic characteristics of individual plants can be extracted. As an example, we show the results for the plant counting and plant height estimation processes where we achieve an accuracy of 88.1% and 89.2%.

Deep neural networks typically require large amounts of annotated data to be trained effectively. However, in several scientific disciplines, including medical image analysis, generating such large annotated datasets requires specialized domain knowledge, and hence is usually very expensive. In this work, we present a novel application of active learning to data sample selection for training Convolutional Neural Networks (CNN) for Cancerous Tissue Recognition (CTR). Our main idea is to steer annotation efforts towards selecting the most informative samples for training the CNN. To quantify informativeness, we explore three choices based on discrete entropy, best-vs-second-best, and k-nearest neighbor agreement. Our results on three different types of cancer datasets consistently demonstrate that under limited annotated samples, our proposed training scheme converges faster than classical randomized stochastic gradient descent, while achieving the same (or sometimes superior) classification accuracy.

Financial and social elements of modern societies are closely connected to the cultivation of corn. Due to the massive production of corn, deficiencies during the cultivation process directly translate to major financial losses. The early detection and treatment of crops deficiencies is thus a task of great significance. Towards an automated health condition assessment, this study introduces a scheme for the computation of plant health indices. Based on the 3D reconstruction of small batches of corn plants, an alternative to existing cumbersome Leaf Area Index (LAI) estimation methodologies is presented. The use of 3D models provides an elevated information content, when compared to planar methods, mainly due to the reduced loss attributed to leaf occlusions. High resolution images of corn stalks are collected and used to obtain 3D models of plants of interest. Based on the extracted 3D point clouds, an accurate calculation of the Leaf Area Index (LAI) of the plants is performed. An experimental validation (using artificially made corn plants used as ground truth of the LAI estimation), emulating real world scenarios, supports the efficacy of the proposed methodology. The conclusions of this work, suggest a fully automated scheme for information gathering in modern farms capable of replacing current labor intensive procedures, thus greatly impacting the timely detection of crop deficiencies.

Computer-Aided Diagnosis (CAD) has witnessed a rapid growth over the past decade, providing a variety of automated tools for the analysis of medical images. In surgical pathology, such tools enhance the diagnosing capabilities of pathologists by allowing them to review and diagnose a larger number of cases daily. Geared towards developing such tools, the main goal of this paper is to identify useful computer vision based feature descriptors for recognizing cancerous tissues in histopathologic images. To this end, we use images of Hematoxylin & Eosin-stained microscopic sections of breast and prostate carcinomas, and myometrial leiomyosarcomas, and provide an exhaustive evaluation of several state of the art feature representations for this task. Among the various image descriptors that we chose to compare, including representations based on convolutional neural networks, Fisher vectors, and sparse codes, we found that working with covariance based descriptors shows superior performance on all three types of cancer considered. While covariance descriptors are known to be effective for texture recognition, it is the first time that they are demonstrated to be useful for the proposed task and evaluated against deep learning models. Capitalizing on Region Covariance Descriptors (RCDs), we derive a powerful image descriptor for cancerous tissue recognition termed, Covariance Kernel Descriptor (CKD), which consistently outperformed all the considered image representations. Our experiments show that using CKD lead to 92.83%, 91.51%, and 98.10% classification accuracy for the recognition of breast carcinomas, prostate carcinomas, and myometrial leiomyosarcomas, respectively.

Analysis of behavior using video is a promising approach for identifying risk markers for psychopathology that can be applied in a wide range of populations. Computer vision techniques are needed to automatically code behavior in order to reduce time and effort in these analyses. This paper discusses algorithms developed for the automatic analysis of video data from a study regarding the impact of environmental factors on youths with obsessive-compulsive disorder. Overhead videos of subjects washing hands were automatically annotated for activities such as rinsing, applying soap, and turning on/off the water faucet. These automated annotations were created by using a color-based background subtraction method to create a foreground probability score which is then used to determine if various labeled regions of interest (ROIs) are activated. These activation signals are then characterized and used to determine when different substeps of the handwashing procedure are performed. Automated annotations were validated by comparisons with hand-labeled ground truth.

Active Constraint Learning (ACL) is continuously gaining popularity in the area of constrained clustering due to its ability to achieve performance gains via incorporating minimal feedback from a human annotator for selected instances. For constrained clustering algorithms, such instances are integrated in the form of Must-Link (ML) and Cannot-Link (CL) constraints. Existing iterative uncertainty reduction schemes, introduce high computational burden particularly when they process larger datasets that are usually present in computer vision and visual learning applications. For scenarios that multiple agents (i.e., robots) require user feedback for performing recognition tasks, minimizing the interaction between the user and the agents, without compromising performance, is an essential task. In this study, a non-iterative ACL scheme with proven performance benefits is presented. We select to demonstrate the effectiveness of our methodology by building on the well known K-Means algorithm for clustering; one can easily extend it to alternative clustering schemes. The proposed methodology introduces the use of the Silhouette values, conventionally used for measuring clustering performance, in order to rank the degree of information content of the various samples. In addition, an efficient greedy selection scheme was devised for selecting the most informative samples for human annotation. To the best of our knowledge, this is the first active constrained clustering methodology with the ability to process computer vision datasets that this study targets. Performance results are shown on various computer vision benchmarks and support the merits of adopting the proposed scheme.

Symmetric Positive Definite (SPD) matrices emerge as data descriptors in several applications of computer vision such as object tracking, texture recognition, and diffusion tensor imaging. Clustering these data matrices forms an integral part of these applications, for which soft-clustering algorithms (K-Means, expectation maximization, etc.) are generally used. As is well-known, these algorithms need the number of clusters to be specified, which is difficult when the dataset scales. To address this issue, we resort to the classical nonparametric Bayesian framework by modeling the data as a mixture model using the Dirichlet process (DP) prior. Since these matrices do not conform to the Euclidean geometry, rather belongs to a curved Riemannian manifold,existing DP models cannot be directly applied. Thus, in this paper, we propose a novel DP mixture model framework for SPD matrices. Using the log-determinant divergence as the underlying dissimilarity measure to compare these matrices, and further using the connection between this measure and the Wishart distribution, we derive a novel DPM model based on the Wishart-Inverse-Wishart conjugate pair. We apply this model to several applications in computer vision. Our experiments demonstrate that our model is scalable to the dataset size and at the same time achieves superior accuracy compared to several state-of-the-art parametric and nonparametric clustering algorithms.

Determining and detecting risk markers for mental illness remains a labor intensive process, requiring vast amounts of observations by clinical professionals. Motor stereotypies, which are defined as involuntary repetitive motor behaviors, invariant in form, that, to an observer, appear to serve no purpose, are a class of risk markers which are very amenable to video analysis. These behaviors are associated with mental illnesses such as Autism, Rett Syndrome, and other developmental disabilities. This paper investigates the application of innovative automated methods to recognize these subtle motor indicators. To train and test our methods, a dataset of actions resembling motor stereotypies was created by engaging the normally developing children at the University of Minnesota's Shirley G. Moore Laboratory School. Comparison to a publicly available dataset depicting a subset of behaviors is performed as well. This work demonstrates the applicability of various techniques in the behavioral science domain. The results show that these techniques can perform well on a difficult and challenging real-world scenario.

The continuously growing need for increasing the production of food and reducing the degradation of water supplies, has led to the development of several precision agriculture systems over the past decade so as to meet the needs of modern societies. The present study describes a methodology for the detection and characterization of Nitrogen (N) deficiencies in corn fields. Current methods of field surveillance are either completed manually or with the assistance of satellite imaging, which offer infrequent and costly information to the farmers about the state of their fields. The proposed methodology promotes the use of small-scale Unmanned Aerial Vehicles (UAVs) and Computer Vision algorithms that operate with information in the visual (RGB) spectrum. Through this implementation, a lower cost solution for identifying N deficiencies is promoted. We provide extensive results on the use of commercial RGB sensors for delivering the essential information to farmers regarding the condition of their field, targeting the reduction of N fertilizers and the increase of the crop performance. Data is first collected by a UAV that hovers over a stressed area and collects high resolution RGB images at a low altitude. A recommendation algorithm identifies potential segments of the images that are candidates exhibiting N deficiency. Based on the feedback from experts in the area a training set is constructed utilizing the initial suggestions of the recommendation algorithm. Supervised learning methods are then used to characterize crop leaves that exhibit signs of N deficiency. The performance of 84.2% strongly supports the potential of this scheme to identify N-deficient leaves even in the case of images where the unhealthy leaves are heavily occluded by other healthy or stressed leaves.

Sparse models have proven to be extremely successful in image processing and computer vision. However, a majority of the effort has been focused on sparse representation of vectors and low-rank models for general matrices. The success of sparse modeling, along with popularity of region covariances, has inspired the development of sparse coding approaches for these positive definite descriptors. While in earlier work, the dictionary was formed from all, or a random subset of, the training signals, it is clearly advantageous to learn a concise dictionary from the entire training set. In this paper, we propose a novel approach for dictionary learning over positive definite matrices. The dictionary is learned by alternating minimization between sparse coding and dictionary update stages, and different atom update methods are described. A discriminative version of the dictionary learning approach is also proposed, which simultaneously learns dictionaries for different classes in classification or clustering. Experimental results demonstrate the advantage of learning dictionaries from data both from reconstruction and classification viewpoints. Finally, a software library is presented comprising C++ binaries for all the positive definite sparse coding and dictionary learning approaches presented here.

Tractor-trailer freight hauling has increased markedly within the United States over the past several years, resulting in higher truck volumes. commercial heavy vehicle drivers are required under federal Hours Of Services rules to rest and take breaks to mitigate driving while fatigued. Although there are many rest area facilities available to truck drivers, there is a lack of persistent timely information on truck parking availability. An automated real-time sensing system to directly detect and disseminate parking space occupancy from truck parking facilities is described in detail. The methodology and system architecture are presented in which robust, persistent, parking occupancy detection is achieved by extending Structure from Motion (SfM) techniques using a multiplicity of commercial, off-the-shelf cameras. The system architecture allows the approach to be scaled to a region-wide comprehensive truck parking information system for commercial heavy vehicle drivers and operators. Per parking space detection accuracy of 99% is achieved over continuous operation. Classification accuracy under diverse scene and parking behavior scenarios is discussed.

In recent years, there has been extensive research on sparse representation of vector-valued signals. In the matrix case, the data points are merely vectorized and treated as vectors thereafter (for example, image patches). However, this approach cannot be used for all matrices, as it may destroy the inherent structure of the data. Symmetric positive definite (SPD) matrices constitute one such class of signals, where their implicit structure of positive eigenvalues is lost upon vectorization. This paper proposes a novel sparse coding technique for positive definite matrices, which respects the structure of the Riemannian manifold and preserves the positivity of their eigenvalues, without resorting to vectorization. Synthetic and real-world computer vision experiments with region covariance descriptors demonstrate the need for and the applicability of the new sparse coding model. This work serves to bridge the gap between the sparse modeling paradigm and the space of positive definite matrices.

This paper presents a new nearest neighbor (NN) retrieval framework: robust sparse hashing (RSH). Our approach is inspired by the success of dictionary learning for sparse coding. Our key idea is to sparse code the data using a learned dictionary, and then to generate hash codes out of these sparse codes for accurate and fast NN retrieval. But, direct application of sparse coding to NN retrieval poses a technical difficulty: when data are noisy or uncertain (which is the case with most real-world data sets), for a query point, an exact match of the hash code generated from the sparse code seldom happens, thereby breaking the NN retrieval. Borrowing ideas from robust optimization theory, we circumvent this difficulty via our novel robust dictionary learning and sparse coding framework called RSH, by learning dictionaries on the robustified counterparts of the perturbed data points. The algorithm is applied to NN retrieval on both simulated and real-world data. Our results demonstrate that RSH holds significant promise for efficient NN retrieval against the state of the art.

Computer vision as an entire field has a wide and diverse range of applications. The specific application for this project was in the realm of dance, notably ballet and choreography. This project was proof-of-concept for a choreography assistance tool used to recognize and record dance movements demonstrated by a choreographer. Keeping the commercial arena in mind, the Kinect from Microsoft was chosen as the imaging hardware, and a pilot set chosen to verify recognition feasibility. Before implementing a classifier, all training and test data was transformed to a more applicable representation scheme to only pass the important aspects to the classifier to distinguish moves for the pilot set. In addition, several classification algorithms using the Nearest Neighbor (NN) and Support Vector Machine (SVM) methods were tested and compared from a single dictionary as well as on several different subjects. The results were promising given the framework of the project, and several new expansions of this work are proposed.

Presented are results demonstrating that, in developing a system with its first objective being the sustained detection of adults and young children as they move and interact in a normal preschool setting, the direct application of the straightforward RGB-D innovations presented here significantly outperforms even far more algorithmically advanced methods relying solely on images. The use of multiple RGB-D sensors by this project for depth-aware object localization economically resolves numerous issues regularly frustrating earlier vision-only detection and human surveillance methods, issues such as occlusions, illumination changes, unexpected postures, atypical morphologies, erratic or unanticipated motions, reflections, and misleading textures and colorations. This multiple RGB-D installation forms the front-end for a multi-step pipeline, the first portion of which seeks to isolate, in situ, 3D renderings of classroom occupants sufficient for a later analysis of their behaviors and interactions. Towards this end, a voxel-based approach to foreground/background separation and an effective adaptation of supervoxel clustering for 3D were developed, and 3D and image-only methods were tested and compared. The project's setting is highly challenging, but then so are its longer term goals: the automated detection of early childhood precursors, ofttimes very subtle, to a number of increasingly common developmental disorders.

Object recognition algorithms often focus on determining the class of a detected object in a scene. Two significant phases are usually involved in object recognition. The first phase is the object representation phase, in which the most suitable features that provide the best discriminative power under constraints such as lighting, resolution, scale, and view variations are chosen to describe the objects. The second phase is to use this representation space to develop models for each object class using discriminative classifiers. In this paper, we focus on composite objects, i.e., objects with two or more simpler classes that are interconnected in a complicated manner. One classic example of such a scenario is a bicyclist. A bicyclist consists of a bicycle and a human who rides the bicycle. When we are faced with the task of classifying bicyclists and pedestrians, it is counterintuitive and often hard to come up with a discriminative classifier to distinguish the two classes. We explore global image analysis based on bag of visual words to compare the results with local image analysis, in which we attempt to distinguish the individual parts of the composite object. We also propose a unified naive Bayes framework and a combined histogram feature method for combining the individual classifiers for enhanced performance.

We study Nearest Neighbors (NN) retrieval by introducing a new approach: Robust Sparse Hashing (RSH). Our approach is inspired by the success of dictionary learning for sparse coding; the key innovation is to use learned sparse codes as hashcodes for speeding up NN. But sparse coding suffers from a major drawback: when data are noisy or uncertain, for a query point, an exact match of the hashcode seldom happens, breaking the NN retrieval. We tackle this difficulty via our novel dictionary learning and sparse coding framework called RSH by learning dictionaries on the robustified counterparts of uncertain data points. The algorithm is applied to NN retrieval for Scale Invariant Feature Transform (SIFT) descriptors. The results demonstrate that RSH is noise tolerant, and at the same time shows promising NN performance over the state-of-the-art.

Object recognition entails extracting information about which object class(es) are present in an image. In order to enhance the performance of object recognition, reducing the redundancy in the data is absolutely essential. Prior literature [1, 2] introduced local and global self-similarity features to highlight the areas in an image which are useful for object classification and detection. We introduce an efficient self-similarity measure based on sparse representations and propose two different descriptors. Our measure of self-similarity is determined across multiple scales and is more efficient than prior work. We test our self similarity descriptor using support vector machine based classification on the PASCAL VOC 2007 database consisting of 20 object classes. Comparative results indicate performance competitive with the prior approaches of computing self-similarity descriptors.

One of the most important tasks for mobile robots is to sense their environment. Further tasks might include the recognition of objects in the surrounding environment. Three dimensional range finders have become the sensors of choice for mapping the environment of a robot. Recognizing objects in point clouds provided by such sensors is a difficult task. The main contribution of this paper is the introduction of a new covariance based point cloud descriptor for such object recognition. Covariance based descriptors have been very successful in image processing. One of the main advantages of these descriptors is their relatively small size. The comparisons between different covariance matrices can also be made very efficient. Experiments with real world and synthetic data will show the superior performance of the covariance descriptors on point clouds compared to state-of-the-art methods.

Clinical studies confirm that mental illnesses such as autism, Obsessive Compulsive Disorder (OCD), etc. show behavioral abnormalities even at very young ages; the early diagnosis of which can help steer effective treatments. Most often, the behavior of such at-risk children deviate in very subtle ways from that of a normal child; correct diagnosis of which requires prolonged and continuous monitoring of their activities by a clinician, which is a difficult and time intensive task. As a result, the development of automation tools for assisting in such monitoring activities will be an important step towards effective utilization of the diagnostic resources. In this paper, we approach the problem from a computer vision standpoint, and propose a novel system for the automatic monitoring of the behavior of children in their natural environment through the deployment of multiple non-invasive sensors (cameras and depth sensors). We provide details of our system, together with algorithms for the robust tracking of the activities of the children. Our experiments, conducted in the Shirley G. Moore Laboratory School, demonstrate the effectiveness of our methodology.

Action classification is an important component of human-computer interaction. Trajectory classification is an effective way of performing action recognition with significant success reported in the literature. We compare two different representation schemes, raw multivariate time-series data and the covariance descriptors of the trajectories, and apply sparse representation techniques for classifying the various actions. The features are sparse coded using the Orthogonal Matching Pursuit algorithm, and the gestures and actions are classified based on the reconstruction residuals. We demonstrate the performance of our approach on standardized datasets such as the Australian Sign Language (AusLan) and UCF Motion Capture datasets, collected using high-quality motion capture systems, as well as motion capture data obtained from a Microsoft Kinect sensor.

Early intervention in mental disorders can dramatically increase an individual's quality of life. Additionally, when symptoms of mental illness appear in childhood or adolescence, they represent the later stages of a process that began years earlier. One goal of psychiatric research is to identify risk-markers: genetic, neural, behavioral and/or social deviations that indicate elevated risk of a particular mental disorder. Ideally, screening of risk-markers should occur in a community setting, and not a clinical setting which may be time-consuming and resource-intensive. Given this situation, a system for automatically detecting risk-markers in children would be highly valuable. In this paper, we describe such a system that has been installed at the Shirley G. Moore Lab School, a research pre-school at the University of Minnesota. This system consists of multiple RGB+D sensors and is able to detect children and adults in the classroom, tracking them as they move around the room. We use the tracking results to extract high-level information about the behavior and social interaction of children, that can then be used to screen for early signs of mental disorders.

In developmental disorders such as autism and schizophrenia, observing behavioral precursors in very early childhood can allow for early intervention and can improve patient outcomes. While such precursors open the possibility of broad and large-scale screening, until now they have been identified only through experts' painstaking examinations and their manual annotations of limited, unprocessed video footage. Here we introduce a system to automate and assist in such procedures. Employing multiple inexpensive real-time rgb+depth (rgb+d) sensors recording from multiple viewpoints, our non-invasive system - now installed at the Shirley G. Moore Lab School, a research preschool - is being developed to monitor and reconstruct the play and interactions of preschoolers. The system's role is to help in assessing the growing volumes of its on-site recordings and to provide the data needed to uncover additional neuromotor behavioral markers via techniques such as data mining.

Covariance matrices of multivariate data capture feature correlations compactly, and being very robust to noise, they have been used extensively as feature descriptors in many areas in computer vision, like, people appearance tracking, DTI imaging, face recognition, etc. Since these matrices do not adhere to the Euclidean geometry, clustering algorithms using the traditional distance measures cannot be directly extended to them. Prior work in this area has been restricted to using K-means type clustering over the Rieman-nian space using the Riemannian metric. As the applications scale, it is not practical to assume the number of components in a clustering model, failing any soft-clustering algorithm. In this paper, a novel application of the Dirich-let Process Mixture Model framework is proposed towards unsupervised clustering of symmetric positive definite matrices. We approach the problem by extending the existing K-means type clustering algorithms based on the logdet divergence measure and derive the counterpart of it in a Bayesian framework, which leads to the Wishart-Inverse Wishart conjugate pair. Alternative possibilities based on the matrix Frobenius norm and log-Euclidean measures are also proposed. The models are extensively compared using two real-world datasets against the state-of-the-art algorithms and demonstrate superior performance.

Sparse models have proven to be extremely successful in image processing and computer vision, and most efforts have been focused on sparse representation of vectors. The success of sparse modeling and the popularity of region covariances have inspired the development of sparse coding approaches for positive definite matrices. While in earlier work [1], the dictionary was pre-determined, it is clearly advantageous to learn a concise dictionary adaptively from the data at hand. In this paper, we propose a novel approach for dictionary learning over positive definite matrices. The dictionary is learned by alternating minimization between the sparse coding and dictionary update stages, and two different atom update methods are described. The online versions of the dictionary update techniques are also outlined. Experimental results demonstrate that the proposed learning methods yield better dictionaries for positive definite sparse coding. The learned dictionaries are applied to texture and face data, leading to improved classification accuracy and strong detection performance, respectively.

Physical constraints that underly the formation of periodic motions can be effectively used to accurately reconstruct the periodic motion from even single camera views. As shown in our earlier work, this reduces to a problem of geometric inference. In this paper, we focus on periodic motions exhibited by humans, which are generally not perfectly periodic, and explore the suitability of the reconstruction techniques in these scenarios. We examine the degree of periodicity of human gait empirically, including the applicability of our motion model. Importantly, we illustrate the usefulness of these techniques by applying them to the task of clinical gait analysis. A computational tool to analyze periodic human motion can prove to be invaluable in medical applications either in terms of assessing deviations from normal patterns or evaluating changes resulting from therapy or other clinical procedures.

Cameras are becoming a common tool for automated vision purposes due to their low cost. In an era of growing security concerns, camera surveillance systems have become not only important but also necessary. Algorithms for several tasks such as detecting abandoned objects and tracking people have already been successfully developed. While tracking people is relatively easy, counting people in groups is much more challenging. The mutual occlusions between people in a group make it difficult to provide an exact count. The aim of this work is to present a method of estimating the number of people in group scenarios. Several considerations for counting people are illustrated in this paper, and experimental results of the method are described and discussed.

Accurate localization of landmarks in the vicinity of a robot is a first step towards solving the SLAM problem. In this work, we propose algorithms to accurately estimate the 3D location of the landmarks from the robot only from a single image taken from its on board camera. Our approach differs from previous efforts in this domain in that it first reconstructs accurately the 3D environment from a single image, then it defines a coordinate system over the environment, and later it performs the desired localization with respect to this coordinate system using the environment's features. The ground plane from the given image is accurately estimated and this precedes segmentation of the image into ground and vertical regions. A Markov Random Field (MRF) based 3D reconstruction is performed to build an approximate depth map of the given image. This map is robust against texture variations due to shadows, terrain differences, etc. A texture segmentation algorithm is also applied to determine the ground plane accurately. Once the ground plane is estimated, we use the respective camera's intrinsic and extrinsic calibration information to calculate accurate 3D information about the features in the scene.

Autonomous estimation of the altitude of an Unmanned Aerial Vehicle (UAV) is extremely important when dealing with flight maneuvers like landing, steady flight, etc. Vision based techniques for solving this problem have been underutilized. In this paper, we propose a new algorithm to estimate the altitude of a UAV from top-down aerial images taken from a single on-board camera. We use a semi-supervised machine learning approach to solve the problem. The basic idea of our technique is to learn the mapping between the texture information contained in an image to a possible altitude value. We learn an over complete sparse basis set from a corpus of unlabeled images capturing the texture variations. This is followed by regression of this basis set against a training set of altitudes. Finally, a spatio-temporal Markov Random Field is modeled over the altitudes in test images, which is maximized over the posterior distribution using the MAP estimate by solving a quadratic optimization problem with L1 regularity constraints. The method is evaluated in a laboratory setting with a real helicopter and is found to provide promising results with sufficiently fast turnaround time.

A framework for robust foreground detection that works under difficult conditions such as dynamic background and moderately moving camera is presented in this paper. The proposed method includes two main components: coarse scene representation as the union of pixel layers, and foreground detection in video by propagating these layers using a maximum-likelihood assignment. We first cluster into "layers" those pixels that share similar statistics. The entire scene is then modeled as the union of such non-parametric layer-models. An in-coming pixel is detected as foreground if it does not adhere to these adaptive models of the background. A principled way of computing thresholds is used to achieve robust detection performance with a pre-specified number of false alarms. Correlation between pixels in the spatial vicinity is exploited to deal with camera motion without precise registration or optical flow. The proposed technique adapts to changes in the scene, and allows to automatically convert persistent foreground objects to background and re-convert them to foreground when they become interesting. This simple framework addresses the important problem of robust foreground and unusual region detection, at about 10 frames per second on a standard laptop computer. The presentation of the proposed approach is complemented by results on challenging real data and comparisons with other standard techniques.

In this work, we propose a framework for foreground representation in video and illustrate it with a multi-camera people matching application. We first decompose the video into foreground and background. A low-level coarse segmentation of the foreground is then used to generate a simple graph representation. A vertex in the graph represents the "appearance" of a corresponding segment in the foreground, while the relationship between two segments is encoded by an edge between the corresponding vertices. This provides a simple yet powerful and general representation of the foreground, which can be very useful in problems such as people detection and tracking. We illustrate the effectiveness of this model using an "example based query" type of application for people matching in videos. Matching results are provided in multiple-camera situations and also under occlusion.

Research in the surveillance domain was confined for years in the military domain. Recently, as military spending for this kind of research was reduced and the technology matured, the attention of the research and development community turned to commercial applications of surveillance. In this paper we describe a state-of-the-art monitoring system developed by a corporate R&D lab in cooperation with the corresponding security business units. It represents a sizable effort to transfer some of the best results produced by computer vision research into a viable commercial product. Our description spans both practical and technical issues. From the practical point of view we analyze the state of the commercial security market, typical cultural differences between the research team and the business team and the perspective of the potential users of the technology. These are important issues that have to be dealt with or the surveillance technology will remain in the lab for a long time. From the technical point of view we analyze our algorithmic and implementation choices. We describe the improvements we introduced to the original algorithms reported in the literature in response to some problems that arose during field testing. We also provide extensive experimental results that highlight the strong points and some weaknesses of the prototype system.

We undertook a study to determine if the automatic detection and counting of vehicle occupants is feasible. In the present paper, we report our findings regarding the appropriate sensor phenomenology and arrangement for the task. We propose a novel system based on fusion of near-infrared imaging signals and demonstrate its adequacy with theoretical and experimental arguments. We also propose a fuzzy neural network classifier to operate upon the fused near-infrared imagery and perform the occupant detection and counting function. We demonstrate experimentally that the combination of fused near-infrared phenomenology and fuzzy neural classification produces a robust solution to the problem of automatic vehicle occupant counting. We substantiate our argument by providing comparative experimental results for vehicle occupant counters based on visible, single near-infrared, and fused near-infrared bands. Our proposed solution can find a more general applicability as the basis for a reliable face detector both indoors and outdoors.

Presents a methodology which allows an autonomous agent i.e., a mobile robot, to learn and build maps of its operating environment by relying only on its range sensors. The maps, described with respect to the robot's inertial frame, are developed in real time by correlating robot position and sensory data. This latter feature characterizes part of the uniqueness of the authors' approach. These maps are topologically isomorphic to the maps created for the same room(s) by humans. The methodology exploits the principle of self-organization, implemented as an artificial neural network module which processes incoming sensor range data. The generation of environmental maps can be visualized as an elastic string of neurons whereby every neuron represents a finite portion of the physical world. This elastic string stretches dynamically so as to take on the shape of the environment, a unique characteristic of the authors' methodology. In this respect, the neural net provides a discretized representation of the "continuous" physical environment as the latter is seen through the robot's own sensors. Experiments, focused on indoor applications, have successfully demonstrated the ability of a robot to build maps of geometrically complex environments. The results presented in this paper, compared with the authors' earlier efforts, show significant improvement in that every single sensor data point contributes equally to the location of the neurons of the spatial map at the end of the learning process. This is important because the authors wish to minimize the effect of the order in which data points are processed.

A methodology is presented whereby a neural network is used to learn the inverse kinematic relationship for a robot arm. A two-link, two-degree-of-freedom planar robot arm is simulated, and an accompanying neural network which solves the inverse kinematic problem is presented. The method is based on Kohonen's self-organizing mapping algorithm using a Widrow-Hoff-type error correction rule as introduced by H. Ritter et al. (1988, 1990). The authors have specifically addressed a number of issues associated with the inverse kinematic solution, including the occurrence of singularities and multiple solutions. Simulation results for a planar two-degree-of-freedom arm provide evidence that this approach is successful. The approach is a significant improvement over other neural net approaches documented in the literature.>