Region Connectivity Calculus (RCC) can be used to define the formal grammar describing the relationship between image regions. While RCC provides a possible framework for representation of object part constellations leading to object recognition, little has been done in the direction of RCC for 3D images. Almost all prior RCC representations, such as RCC5/ RCC8/ RCC23/RCC62 are oriented towards 2D projections. While the recently introduced RCC-3D does address this limitation to a certain extent, it heavily depends on other 2D RCC frameworks and as such is limited -- it provides no representation for orthogonally aligned or staggered object parts. It also does not provide convenient representations for shape related pose information (such as "horizontally aligned along principal axis" or "vertically aligned" etc.). While it is possible to use oriented matroids for projected 3D representations using cocircuits and chirotopes, these are again sub-optimal given that there is considerable information loss (through dimensionality reduction) and multiple object models map to the same structures. In this paper, we introduce a new hierarchical graph based 3D Region/ Surface/ Object/ Part Connectivity Calculus (OCC/PCC), given the domain of affordance based equivalence recognition. We call our PCC -- 4D Space Time Mereo-topo-geometry (4D-STMTG) and the equivalent OCC as 4D Space Time Joint Topo-geometry (4D-STJTG). The modeling of simple objects using the calculus is demonstrated in practical scenarios. Comparisons of the proposed calculus with respect to the state-of-art RCC-3D is also presented, demonstrating the flexibility, suitability and superiority of 4D-STMTG.

The task of searching and grasping objects in cluttered scenes, typical of robotic applications in domestic environments requires fast object detection and segmentation. Attentional mechanisms provide a means to detect and prioritize processing of objects of interest. In this work, we combine a saliency operator based on symmetry with a segmentation method based on clustering locally planar surface patches, both operating on 2.5D point clouds (RGB-D images) as input data to yield a novel approach to table-top scene segmentation. Evaluation on indoor table-top scenes containing man-made objects clustered in piles and dumped in a box show that our approach to selection of attention points significantly improves performance of state-of-the-art attention-based segmentation methods.

Compressed Sensing (CS) is seen as the pathway to increase the efficiency of sensor systems such as MRI, SAR and SAS while avoiding the huge costs and related processing accompanying high-resolution data acquisition. While there has been a surge in the number of sensor systems and related algorithms using CS, target/object recognition in the sensing domain which offers numerous advantages, is a rather nascent field. The state-of-the-art in this field includes the Smashed Filter (SF), which is a reduced dimensionality maximum likelihood classifier. Nevertheless, the accuracy of the filter remains low for practical applications, especially with variations in scale, translation and rotation in the test data. This paper offers a new type of filter - called the Compressive Distance Classifier Correlation Filter (CDCCF), which applies a transformation in the CS domain thereby increasing the distance between intra-class correlation peaks while reducing the distance between inter-class correlation peaks and is based on the Restricted Isometry Property (RIP) of the compressed manifold and the Johnson Lindenstrauss Lemma. Results presented show that the accuracy of the CDCCF filter is about 70% on a 12 class test data set, which is over a two-fold increase in accuracy over the SF. Confusion matrices, measures of ROC, Mean Average Precision and Accuracy demonstrate the robust performance of the algorithm over SF across different compressive sampling resolutions.

Model fitting is a fundamental component in computer vision for salient data selection, feature extraction and data parameterization. Conventional approaches such as the RANSAC family show limitations when dealing with data containing multiple models, high percentage of outliers or sample selection bias, commonly encountered in computer vision applications. In this paper, we present a novel model evaluation function based on Gaussian-weighted Jensen–Shannon divergence, and integrate into a particle swarm optimization (PSO) framework using ring topology. We avoid two problems from which most regression algorithms suffer, namely the requirements to specify inlier noise scale and the number of models. The novel evaluation method is generic and does not require any estimation of inlier noise. The continuous and meta-heuristic exploration facilitates estimation of each individual model while delivering the number of models automatically. Tests on datasets comprised of inlier noise and a large percentage of outliers (more than 90 % of the data) demonstrate that the proposed framework can efficiently estimate multiple models without prior information. Superior performance in terms of processing time and robustness to inlier noise is also demonstrated with respect to state of the art methods.

Generation of grasps for automated object manipulation in cluttered scenarios presents major challenges for various modules of the pipeline such as 2D/3D visual processing, 3D modeling, grasp hypothesis generation, grasp planning and path planning. In this paper, we present a solution framework for solving a complex instance of the problem - represented by a heap of unknown and unstructured objects in a bounded environment - in our case, a box; with the goal being removing all objects in the box using an attention driven object modeling approach to cognitive grasp planning. The focus of the algorithm delves on Grasping by Components (GBC), with a prioritization scheme derived from scene based attention and attention driven segmentation. In order to overcome the traditional challenge of segmentation performing poorly in cluttered scenes, we employ a novel active segmentation approach suited to our scenario. While the attention module helps prioritize objects in the heap and salient regions, the GBC scheme segments out parts and generates grasp hypotheses for each part. GBC is a very important component of any scalable and holistic grasping system since it abstracts point cloud object data with parametric shapes and no apriori knowledge (such as 3D models) is required. Earlier work in 3D model building (such as CAD based, simple geometries, bounding boxes, Superquadrics etc.) have depended on precise shape and pose recognition as well as exhaustive training to learn or exhaustive searching in grasp space to generate good grasp hypotheses. These methods are not scalable for real-time scenarios, complex shapes and unknown environments - key challenges in robotic grasping. In order to alleviate this concern, we present a novel parametric algorithm to estimate grasp points and approach vectors from the 3D parametric shape model, along with innovative schemes to optimize the computation of the parametric models as well as to refine the generated grasp hypotheses based on the scene information to aid path planning. We present evaluation of our complex grasping pipeline for cluttered heaps through a series of test sequences involving removal of objects from a box, along with evaluations for our attention mechanisms, active segmentation, 3D model fitting optimizations and quality of our grasp hypotheses.

AfNet, The Affordance Network is an open affordance computing initiative that provides affordance knowledge ontologies for common household articles in terms of affordance features using surface forms termed as afbits (affordance bits). AfNet currently offers 68 base affordance features (25 structural, 10 material, 33 grasp), providing over 200 object category definitions in terms of 4000 afbits. Symbol grounding algorithms for these affordance features enable recognition of objects in visual (RGB-D) data. While AfNet is built as a generic visual knowledge ontology for recognition, it is well suited for deployment on domestic robots. In this paper, we describe AfRob, an extension of AfNet for robotic applications. AfRob builds upon AfNet by imbibing semantic context and mapping for holistic recognition and manipulation of objects in domestic environments. AfRob also offers modules to enable robots to interact and grasp objects through the generation of grasp affordances. The paper also details the inference mechanisms that adapt AfNet for robots in domestic contexts. Results demonstrate the efficiency of the affordance driven approach to holistic visual processing.

Traffic psychology presents interesting avenues towards the development of Intelligent Transportation Systems (ITS). Analysis of driver state, emotion and behavior are important components of traffic psychology. While being comprehensive in terms of theoretical frameworks, these analyses lack neurobiological computational models for evaluation. In this paper, we develop computational models for driver state and behavior, also known as Mental State Inference (MSI) based on the Mirror Neuron System (MNS) architecture. The integrated system combines neurobiological models with computer vision techniques for traffic monitoring from surveillance video leading to MSI and event recognition. Evaluation of the system is carried out in terms of actual, psychophysical as well as neurobiological criteria on both simulated and real data. Results demonstrate event and mental state recognition convergence within 0.5 normalized time units for the designed event models on synthetic and real data. The model is also robust to perturbations and is aligned to behavior expected at the psychophysical level.

Range data obtained from conventional stereo-cameras employing dense stereo matching algorithms typically contain a high amount of noise, especially under poor illumination conditions. Furthermore, lack of reliable depth estimates in low-texture regions can result in poor 3D surface reconstruction. Anisotropic diffusion algorithms have been used recently in stereo matching, depth estimation and 3D surface reconstruction. However, these algorithms typically have long execution times, preventing real-time operation on resource constrained systems and robots. Moreover, most of these techniques suffer from excessive smoothing at depth discontinuities resulting in loss of structure, especially in areas where the 2D image does not provide structural cues to guide the depth diffusion. These algorithms are also unsuitable for diffusion of extremely sparse depth data such as in the case of homogenous surfaces. This paper addresses these issues by novel denoising and diffusion techniques. The results presented demonstrate the run-time efficiency and fidelity of reconstructed depth surfaces.

Traditional indoor 3D structural environment modeling algorithms employ schemes such as clustering of dense point clouds for parameterization and identification of the 3D surfaces. RANSAC based plane fitting is one common approach in this regard. Alternatively, extensions to feature based stereo have also been used, mainly focusing on 3D line descriptions, along with techniques such as half-plane detection, real-plane or facade reconstruction, plane sweeping etc. Noise in the range data, especially in low texture regions, accidental line/plane grouping under lack of cues for visibility tests, presence of depth edges or discontinuities that are not visible in the 2D image and difficulties in adaptively estimating metrics for clustering can hamper efficiency of practical systems. In order to counter these issues, we propose a novel framework fusing 2D local and global features such as edges, texture and regions, with geometry information obtained from range data for reliable 3D indoor scene representation. The strength of the approach is derived from the novel depth diffusion and segmentation algorithms resulting in superior surface characterization as opposed to traditional feature based stereo or RANSAC based plane fitting approaches. These algorithms have also been heavily optimized to enable real-time deployments on personal, domestic and rehabilitation robots.