Plane detection from depth images is a crucial subtask with broad robotic applications, often accomplished by iterative methods such as Random Sample Consensus (RANSAC). While RANSAC is a robust strategy with strong probabilistic guarantees, the ambiguity of its inlier threshold criterion makes it susceptible to false positive plane detections. This issue is particularly prevalent in complex real-world scenes, where the true number of planes is unknown and multiple planes coexist. In this paper, we aim to address this limitation by proposing a generalised framework for plane detection based on model information optimization. Building on previous works, we treat the observed depth readings as discrete random variables, with their probability distributions constrained by the ground truth planes. Various models containing different candidate plane constraints are then generated through repeated random sub-sampling to explain our observations. By incorporating the physics and noise model of the depth sensor, we can calculate the information for each model, and the model with the least information is accepted as the most likely ground truth. This information optimization process serves as an objective mechanism for determining the true number of planes and preventing false positive detections. Additionally, the quality of each detected plane can be ranked by summing the information reduction of inlier points for each plane. We validate these properties through experiments with synthetic data and find that our algorithm estimates plane parameters more accurately compared to the default Open3D RANSAC plane segmentation. Furthermore, we accelerate our algorithm by partitioning the depth map using neural network segmentation, which enhances its ability to generate more realistic plane parameters in real-world data.

Deep reinforcement learning (DRL) frameworks have shown their remarkable effectiveness in learning navigation policy for the mobile robot navigating in a human crowded environment. Moreover, attention mechanisms coupled with DRL allows the robot to identify neighbors with different level of influence and incorporate them into the robot’s decision. However, as the crowd density increases, attention mechanisms may fail to identify critical neighbors which can lead to significant drops in navigation efficiency. In this work, we aim to address this limitation by encoding both human-human and human-robot interaction using a special class of Graph Convolutional Networks (GCN) known as Message-Passing GCN (MP-GCN). In contrast to existing methods, where attention between robot and humans are encoded uniformly, the proposed approach named MP-GatedGCN-RL encodes asymmetric interactions using the combination of novel message-passing function and edge-wise gating mechanisms. We evaluate our approach on the simulated environments of ETH/UCY pedestrians datasets consisting of different scenarios like collision avoidance, group forming, diverging, crossing, and so on. Experimental results demonstrate that our proposed method outperforms the conventional benchmark dynamic avoidance method ORCA with a 20.6% increase in success rate and a 9.1% reduction in navigation time. Moreover, we also achieve a 5.5% enhancement in success rate compared to other state-of-the-art DRL-based methods without any additional labeled expert data nor prior supervised learning.

The recent years have seen the increasing importance of cognitive models for improved robot navigation. In this paper, a novel cognitive navigation package, which consists of topometric map representation and a three-level path planner, is proposed. The topometric maps are built from architectural floor plans with additional features within a limited number of selected regions. The inherent discrepancies between floor plans and the robot’s actual sensory readings are handled by the three-level path planner. The unique feature of this approach is that accurate localization is only required at the selected regions. At the other regions the robot will rely on the guiding directions towards the goal rather than on its accurate position on the map for navigation. Experiments show that our approach can endow a robot with capability for semantic interpretation and localization in unseen and dynamic environments.

Multi-human tracking in the crowded environment is a challenging problem due to occlusions, pose change, viewpoint variation and cluttered background. In this work, we propose a robust feature learning for tracking-by-detection methods based on second-order attention network that can capture higher-order relationships between salient features at the early stages of Convolutional Neural Network (CNN). Guided Second-Order Attention Network (GSAN) that, unlike the existing attention learning methods which are weakly-supervised, uses a supervisory signal based on the quality of the self-learned attention maps. More specifically, GSAN looks into the attended maps of a person having the highest confidence and supervise itself to look into the correct regions in the images of the person. Attention maps learned this way are spatially aligned and thus robust to camera-view changes and body pose variations. We verify the effectiveness of our approach by comparing with the state-of-the-art methods on challenging person re-identification and multi object tracking (MOT) datasets.

In scenarios where the environment and task are known and certain, autonomous robotic manipulation can be performed. However, when an unforeseen situation arises where prior information is unknown, timely human assistance becomes very useful in accomplishing the task. In this paper, we propose a supervisory autonomy system that allows a user to assist the robot in uncertain task scenarios by specifying, through an intuitive user interface, the discrete actions (e.g. grasping) to be performed on the selected objects. The robot has partial knowledge of the environment in that it is able to segment and localize objects. This information is shared with the user to allow easy object and action specification, after which the robot executes the action on the object autonomously while avoiding obstacles. We investigated the system using live perception of real objects with simulated robot and environment.

Humans are adept at tool use. We can intuitively and immediately improvise and use unknown objects in our environment as tools, to assist us in performing tasks. In this study, we provide similar cognition and capabilities to robots. Neuroscientific studies on tool use have suggested that human dexterity with tools is enabled by the embodiment of the tools, which in effect, allows humans to immediately transfer prior skills acquired without tools, onto tasks requiring tool use. Here, utilizing the theoretical results from our investigations on embodiment and tool use in humans over the last years, we propose a concept and algorithm to enable similar skill transfer by robots. Our algorithm enables a robot that has had no prior learning with tools, to automatically recognize an object (seen for the first time) in its environment as a potential tool for an otherwise unattainable task, and use the tool to perform the task thereafter.

Recognition of natural emotions from human faces is an interesting topic with a wide range of potential applications, such as human-computer interaction, automated tutoring systems, image and video retrieval, smart environments, and driver warning systems. Traditionally, facial emotion recognition systems have been evaluated on laboratory controlled data, which is not representative of the environment faced in real-world applications. To robustly recognize the facial emotions in real-world natural situations, this paper proposes an approach called extreme sparse learning, which has the ability to jointly learn a dictionary (set of basis) and a nonlinear classification model. The proposed approach combines the discriminative power of extreme learning machine with the reconstruction property of sparse representation to enable accurate classification when presented with noisy signals and imperfect data recorded in natural settings. In addition, this paper presents a new local spatio-temporal descriptor that is distinctive and pose-invariant. The proposed framework is able to achieve the state-of-the-art recognition accuracy on both acted and spontaneous facial emotion databases.

In this paper, a novel vision system is proposed to estimate attention of people from rich visual clues for social robot to perform natural interactions with multiple participants in public environments. The vision detection and recognition modules include multi-person detection and tracking, upper-body pose recognition, face and gaze detection, lip motion analysis for speaking recognition, and facial expression recognition. A computational approach is proposed to generate a quantitative estimation of human attention. The vision system is implemented on a robotic receptionist "EVE" and encouraging results have been obtained.

Although facial expression plays an important role in human interaction, automated facial expression analysis is still a challenging task. This paper presents a novel facial descriptor based on Phase Congruency (PC) and Local Binary Pattern (LBP) for facial expression recognition. The proposed descriptor, named Binary Pattern of Phase Congruency (BPPC), is an oriented and multi-scale local descriptor that is able to encode various patterns of face images. It is constructed by applying LBP on the oriented PC images. We evaluated the proposed method using the Cohn-Kanade (CK+) database. In our experiment, we achieved an overall detection rate of 93.83% for the six basic emotions. This shows the effectiveness of the proposed method.

Abstract A critical issue in the development of an automated fingerprint identification system (AFIS) is to detect the reference point accurately and reliably. Most existing techniques are based on the singular points of the fingerprint which is sensitive to fingerprint artifacts and is not well defined for arch type fingerprints. In this study, we present a topological interpretation of fingerprint reference point, where the reference points are posed as topological features of fingerprint structure and can be detected seamlessly from either arch type or non‐arch type fingerprints. Extensive experiments demonstrate the advantages of the proposed method over the existing ones. Copyright © 2010 John Wiley & Sons, Ltd.

In this paper, a novel face representation in terms of dense local image descriptors is proposed. Scale Invariant Feature Transform (SIFT) and Gabor, two of the most popular local image descriptors, at dense grid pixels of a face image are used to represent the face. The efficiency of the representation has been investigated in gender recognition. There are four problems when applying the SIFT to facial gender recognition. (1) There may be only a few keypoints that can be found in a face image due to the missing texture and ill-illumined faces; (2) The SIFT descriptors at the keypoints (we called it sparse SIFT) are distinctive whereas alternative descriptors at non-keypoints (e.g. grid) could cause negative impact on the accuracy; (3) Most of the existing methods employ SIFT descriptors matching in which relative larger image size is required in order that enough keypoints can be found to support the matching and (4) The matching is assumed that the faces are well registered. We provide solutions to the above difficulties in this paper and the problem of recognizing gender using the combination of SIFT descriptors and Gabor of face images is studied. The Gabor representations of the face images are fused with the dense SIFT at the feature-level to improve the accuracy. AdaBoost is adopted to select features and form a strong classifier. The experimental results on a large set of faces have shown that the proposed method can achieve high accuracies even for faces that are not aligned.

This paper presents a novel method for fingerprint orientation modeling, which executes in two phases. Firstly, the orientation field is reconstructed through fitting to a lower order Legendre polynomial basis to capture the global orientation pattern. Then the preliminary model around the singular region is dynamically refined by fitting to a higher order Legendre polynomial basis. The singular region is automatically detected through the analysis on the orientation residual field between the original orientation field and the orientation model. The method has been evaluated using the FVC 2004 data sets and compared with state-of-the-arts. Experiments turn out that the propose method attains higher accuracy in fingerprint matching and singularity preservation.

Singular point detection is an important issue in fingerprint image analysis. General methods like Poincare index method can detect singular points in non-arch type fingerprints but fail on arch-type fingerprints. Some more sophisticated methods like complex filter method also face the same problem. In this paper, we propose a systematic method for detecting singular points in fingerprint images which utilizes the most fundamental topological feature of acquired fingerprints as the basis for singular point identification. The method differentiates the input fingerprint between arch type and non-arch type. For non-arch type fingerprints singular points are detected as intersection points in a c(>2) level segmentation map. As for arch type fingerprints, singular points are identified from the symmetric line of the fingerprint structure. The method is evaluated using the NIST DB4 database and compared with the complex filter method. The proposed method attains near 90% success rate in detecting singular points and the displacement from ground truth is comparable to that of the complex filter method. Moreover our method is more than once faster in CPU time.

This paper proposes to characterize the fingerprint orientation using a novel topological representation, which transforms the orientation field into a map composed of a set of geometric objects and analyzes the properties of each geometric object based on two assumptions: regional coherence assumption and convexity assumption. Different from prior works on fingerprint orientation analysis, this approach has a capability to identify where the errors happen, not only the ldquorandom errorsrdquo (which is assumed by local-filtering and global modeling methods), but also the ldquostructure irregularitiesrdquo, which often occur in real prints and impose difficulty on existing techniques.

In this paper, we proposed, a new approach to track the object of interest using bi-model algorithm. The bi-model comprises a long term model and a short term model to allow continuous tracking of the object of interest. To measure the similarity of the bi-model and the candidate targets, an invariant spatiogram descriptor was proposed. Compared to the original spatiogram the new definition is invariant to rotation without sacrificing the performance. The experiments on our dataset and the publicly available datasets verify the effectiveness of our proposed algorithm.

In a large number of fingerprint database, the naïve way to identify a person is to match the query image with all entries of the database. To speed up this processing, exclusive classification assigns each fingerprint into one of several subgroups. Thus the query image only needs to compare with the images in one subgroup. While exclusive classification is hard because of small inter-class variability and the large intra-class variability. And it is not so efficient as well due to the small number of classes and non-uniform distribution.

Singular point is crucial for fingerprint identification. However, detection of the singular point is not always perfect. Up to now, there is no effective criterion to validate the singular point detected. This paper introduces a new interactive mechanism to detect and validate a singular point. The method comprises two stages: 1. the candidate singular point is located using the complex symmetrical filters. The magnitude of the candidate singular point represents the singularity information; 2. For each detected singular point, a model-based method is used to reconstruct the orientation near the candidate point. The reconstructed orientation is used to represent its orientation information. By combining both singularities and computing the difference between the reconstructed orientation and the original orientation, the candidate singular point can be validated. Experimental results show that this method can detect the singular point even with partially corrupted orientation pattern and remove falsely detected singular points

Fingerprint orientation is crucial for automatic fingerprint identification. However, recovery of orientation is still difficult especially in noisy region. A way to aid recovery of the orientation is to provide an orientation model. In this paper, an orientation model for the entire fingerprint orientation using high order phase portrait is suggested. Proper analysis of the orientation pattern at the singular point regions is provided. Then a low-order phase portrait near each of the singular point is added as constraint to the high-order phase portrait to provide accurate orientation modeling for the entire fingerprint image. The main advantage of the proposed approach is that the nonlinear model itself is able to model all type of fingerprint orientations completely. Experiments and visual analysis show the effectiveness of the proposed model.