The importance of face re-identification lies in its ability to provide a consistent and reliable means of identifying individuals. Unlike attributes such as clothing or gait, which can change frequently, facial features serve as a more stable identifier. On the other hand, synthetic data generation has emerged as a valuable tool in facial re-identification technologies. This study proposes the use of generative models to create synthetic images that enrich the representation of each identity and improve re-identification tasks. To achieve this, we designed various facial re-identification configurations based on the LFW dataset. We focused on a setup where both the gallery and the probe sets are expanded with synthetic data. The results showed that the combination of embeddings obtained from synthetic images generated by generative models provides a richer and more robust identity representation than embeddings derived solely from the original image, achieving an improvement of 17 points in Rank-1 accuracy and an improvement of 15.2% in mAP. Furthermore, the use of LCM-LoRA acceleration in the process reduced the times by 10x, without compromising quality in re-identification tasks.

Recognition of human actions is a very important task in computer vision research. It can be used for robots, to better understand human companions; by surveillance systems, to detect suspicious action; or even in health care setups to detect falls or other harming events. Despite the fact that action recognition is a recurrent topic, it still poses a significant challenge. In this work, we present HARTO, a novel optical flow-based deep learning architecture for classifying human action. The input of the network is an array of optical flow frames extracted from a video. This intermediate representation allows us to optimally encode the motion present in videos to provide accurate predictions. We trained and tested our approach with the Toyota Smarthome Dataset, which is a commonly used benchmark for this task. We follow the protocols proposed by the creators of the dataset and ranked #3 on a public benchmark at the time of writing.

In recent years, the popularity of intelligent and autonomous vehicles has grown notably. In fact, there already exist commercial models with a high degree of autonomy as regards self-driving capabilities. A key feature for this kind of vehicle is object detection, which is commonly performed in 2D space. This has some inherent issues as an object and the depiction of such an object would be classified as the actual object, which is inadequate since urban environments are full of billboards, printed adverts and posters that would likely make these systems fail. In order to overcome this problem, a 3D sensor could be leveraged, although this would make the platform more expensive, energy inefficient and computationally complex. Thus, we propose the use of structure from motion to reconstruct the three-dimensional information of the scene from a set of images, and merge the 2D and 3D data to differentiate actual objects from depictions. As expected, our approach is able to work with a regular color camera. No 3D sensors whatsoever are required. As the experiments confirm, our approach is able to distinguish between actual pedestrians and depictions of them more than 87% of times in synthetic and real-world tests in the worst scenarios, while the accuracy is of almost 98% in the best case.

In three-dimensional data, despite deep learning-based approaches achieve decent accuracy when applied to classification, they struggle on other more complex tasks such as registration. This is why traditional keypoint detection, description and matching pipelines are even currently state-of-the-art. In this paper we propose VFKD, a Voxelized Fractal Keypoint Detector. We leverage the fractal dimension of the neighborhood of a point to either classify it as a keypoint or not. Our approach accepts a set of parameters which should be tuned for each final application, thus providing high versatility. VFKD outperforms state-of-the-art 3D keypoint extractors in terms of mean displacement error by a large margin.

Building upon the recent progress in novel view synthesis, we propose its application to improve monocular depth estimation. In particular, we propose a novel training method split in three main steps. First, the prediction results of a monocular depth network are warped to an additional view point. Second, we apply an additional image synthesis network, which corrects and improves the quality of the warped RGB image. The output of this network is required to look as similar as possible to the ground-truth view by minimizing the pixel-wise RGB reconstruction error. Third, we reapply the same monocular depth estimation onto the synthesized second view point and ensure that the depth predictions are consistent with the associated ground truth depth. Experimental results prove that our method achieves state-of-the-art or comparable performance on the KITTI and NYU-Depth-v2 datasets with a lightweight and simple vanilla U-Net architecture.

There is no doubt that the use of machine learning is increasing every day. Its applications include self-driving cars, malware detection, recommendation systems and many other fields. Although the broad scope of this technology highlights the importance of its reliability, it has been shown that machine learning models can be vulnerable to adversarial attacks. In this paper, we study a property of these attacks called transferability across different architectures and models, measuring how these attacks transfer based on a specific number of parameters among three adversarial attacks: Fast Gradient Sign Method, Projected Gradient Descent and HopSkipJumpAttack.

With the emergence of low cost 3D sensors, the focus is moving towards the recognition and scene understanding of tridimensional data. This kind of representation is really challenging in terms of computation, and it needs the development of new strategies and algorithms to be handled and interpreted.In this work, we propose NurbsNet, a novel approach for 3D object classification based on local similarities with free form surfaces modeled as Nurbs.The proposal has been tested in ModelNet10 and ModelNet40 with results that are promising with less training iterations than state-of-the-art methods and very low memory consumption.

Self-driving cars are becoming slowly but surely the future of transport. Nonetheless, in order to achieve fully automatic operation, several challenges are still needed to be tackled. One of the main goals that is currently being pursued is a very accurate scene understanding and object detection. In this regard, the most accurate object detectors are image-based. However, these methods yield critical flaws that make them prone to error in some specific scenarios. For instance, actual objects would be detected in depictions of such objects. The urban environment is strewn with these cases. Namely, in billboards and advertisements. However, most of the self-driving cars feature a lidar that provides 3D perception. This sensor could help to disambiguate the cases mentioned before.In this paper, we combine the accuracy of 2D deep learning object detectors with a 3D Convolutional Neural Network (3D CNN) for rejecting false positives on pedestrian detection. First, the object detector provides all the detected pedestrians in the scene, and then the 3D CNN is in charge of rejecting or verify the detections. Our proposal is tested on two well-known publicly available datasets and provides up to 84% accuracy.

As estimated by the World Health Organization, there are millions of people who lives with some form of vision impairment . As a consequence, some of them present mobility problems in outdoor environments . With the aim of helping them, we propose in this work a system which is capable of delivering the position of potential obstacles in outdoor scenarios. Our approach is based on non-intrusive wearable devices and focuses also on being low-cost. First, a depth map of the scene is estimated from a color image, which provides 3D information of the environment. Then, an urban object detector is in charge of detecting the semantics of the objects in the scene. Finally, the three-dimensional and semantic data is summarized in a simpler representation of the potential obstacles the users have in front of them. This information is transmitted to the user through spoken or haptic feedback. Our system is able to run at about 3.8 fps and achieved a 87.99% mean accuracy in obstacle presence detection. Finally, we deployed our system in a pilot test which involved an actual person with vision impairment, who validated the effectiveness of our proposal for improving its navigation capabilities in outdoors.

In the last years, the care of dependent people, either by disease, accident, disability, or age, is one of the current priority research topics in developed countries. Moreover, such care is intended to be at patients home, in order to minimize the cost of therapies. Patients rehabilitation will be fulfilled when their integration in society is achieved, either in the family or in a work environment. To address this challenge, we propose the development and evaluation of an assistant for people with acquired brain injury or dependents. This assistant is twofold: in the patient’s home is based on the design and use of an intelligent environment with abilities to monitor and active learning, combined with an autonomous social robot for interactive assistance and stimulation. On the other hand, it is complemented with an outdoor assistant, to help patients under disorientation or complex situations. This involves the integration of several existing technologies and provides solutions to a variety of technological challenges. Deep leaning-based techniques are proposed as core technology to solve these problems.

Biomarker estimation methods from medical images have traditionally followed a segment-and-measure strategy. Deep-learning regression networks have changed such a paradigm, enabling the direct estimation of biomarkers in databases where segmentation masks are not present. While such methods achieve high performance, they operate as a black-box. In this work, we present a novel deep learning network structure that, when trained with only the value of the biomarker, can perform biomarker regression and the generation of an accurate localization mask simultaneously, thus enabling a qualitative assessment of the image locus that relates to the quantitative result. We showcase the proposed method with three different network structures and compare their performance against direct regression networks in four different problems: pectoralis muscle area (PMA), subcutaneous fat area (SFA), liver mass area in single slice computed tomography (CT), and Agatston score estimated from non-contrast thoracic CT images (CAC). Our results show that the proposed method improves the performance with respect to direct biomarker regression methods (correlation coefficient of 0.978, 0.998, and 0.950 for the proposed method in comparison to 0.971, 0.982, and 0.936 for the reference regression methods on PMA, SFA and CAC respectively) while achieving good localization (DICE coefficients of 0.875, 0.914 for PMA and SFA respectively, p < 0.05 for all pairs). We observe the same improvement in regression results comparing the proposed method with those obtained by quantify the outputs using an U-Net segmentation network (0.989 and 0.951 respectively). We, therefore, conclude that it is possible to obtain simultaneously good biomarker regression and localization when training biomarker regression networks using only the biomarker value.

Most state-of-the-art machine learning classifiers currently perform a direct classification of categories. Nonetheless, all the concepts are inherently connected by an inclusion dependency, creating a tree of relationships. For example, a car and a bus are elements included in the concept vehicle. Traditional approaches make use of finer grain concepts, thus discarding important data provided by these relational bounds. This paper describes different strategies to include this tree information in a deep learning pipeline. We compare our proposals, which produce multilevel classifications, with a direct classification baseline, outperforming it in each case. In addition, multilevel predictions could be further improved with different path correction strategies, which were also tested.

For a robot, the ability to adapt his knowledge automatically and customize its behavior is a key feature. Furthermore, a robot should be able to carry out its tasks at a long-term basis, performing it seamlessly in presence of changes in their surroundings. To do that, it is essential that the robot dynamically learn from their environment, but to perform a fully retraining of a deep learning architecture when the model needs new knowledge is a highly time consuming task. This work focus on exploring several strategies to include new data to an already learned model, applied to the semantic localization problem focusing in the accuracy of the final model and their training time. Exhaustive experimentation is carried out and each result is discussed consequently.

In the last years, we have seen a large growth in the number of applications which use deep learning-based object detectors. Autonomous Driving Assistance Systems (ADAS) is one of the areas where it has more impact. In this work, we present a novel study that evaluates a state-of-the-art technique for urban object localization. In particular, we investigate the performance of the Faster R-CNN method to detect and localize urban objects in a variety of outdoor urban videos involving pedestrians, cars, bicycles and other objects moving in the scene. We propose a new dataset that is used for benchmarking the accuracy of a real-time object detector (Faster R-CNN). Part of the data was collected using an HD camera mounted in a vehicle. Besides, some of the data is weakly annotated so it can be used for testing weakly-supervised learning techniques. We have carried out extensive experiments demonstrating the effectiveness of the baseline approach, which achieved a 74.2% accuracy on the proposed dataset. Moreover, we have evaluated a baseline approach for traffic sign recognition achieving an accuracy of 98.1%. A ResNet-based architecture was trained and used for this purpose as a second stage of our object detector. The full dataset is available for download at http://www.rovit.ua.es/dataset/traffic/.

In the last few years, Convolutional Neural Networks (CNNs) had become the default paradigm to address classification problems, specially, but not only, in image recognition. This is mainly due to the high success rate that they provide. Despite there currently exist approaches that apply deep learning to the 3D recognition problem, they are either too slow for online uses or too error prone. To fill this gap, we propose LonchaNet, a deep learning architecture for point clouds classification. Our system successfully achieves a high accuracy yet providing a low computation cost. A dense set of experiments were carried out in order to validate our system in the frame of the ModelNet - a large-scale 3D CAD models dataset - challenge. Our proposal achieves a success rate of 94.37% in the ModelNet-10 classification task, the second place in the leaderboard as of today (November, 2016).

Schaeffer language is considered an effective method to help autistic children overcome communicative disorders. Speech and language therapy results in an improvement in communication skills and understanding of language productions. In this work, a Schaeffer language recognition system is presented with the purpose of teaching children with autism disorder the correct way to communicate using gestures in combination with speech reproduction. The purpose is to accelerate the learning process and increase children interest using a technological approach. A Long Short-Term Memory (LSTM) model has been implemented for this purpose reporting a 93.13% classification success rate over a subset of 25 Schaeffer gestures. A comparison with vanilla RNNs and GRU-based models has been also carried out. Pose-based features such as angles and euclidean distances have been extracted from our gesture dataset by processing raw skeletal data from a Kinect v2 sensor.

Pedestrian movement direction recognition is an important factor in autonomous driver assistance and security surveillance systems. Pedestrians are the most crucial and fragile moving objects in streets, roads, and events, where thousands of people may gather on a regular basis. People flow analysis on zebra crossings and in shopping centers or events such as demonstrations are a key element to improve safety and to enable autonomous cars to drive in real life environments. This paper focuses on deep learning techniques such as convolutional neural networks (CNN) to achieve a reliable detection of pedestrians moving in a particular direction. We propose a CNN-based technique that leverages current pedestrian detection techniques (histograms of oriented gradients-linSVM) to generate a sum of subtracted frames (flow estimation around the detected pedestrian), which are used as an input for the proposed modified versions of various state-of-the-art CNN networks, such as AlexNet, GoogleNet, and ResNet. Moreover, we have also created a new data set for this purpose, and analyzed the importance of training in a known data set for the neural networks to achieve reliable results.

During the last few years, Convolutional Neural Networks are slowly but surely becoming the default method solve many computer vision related problems. This is mainly due to the continuous success that they have achieved when applied to certain tasks such as image, speech, or object recognition. Despite all the efforts, object class recognition methods based on deep learning techniques still have room for improvement. Most of the current approaches do not fully exploit 3D information, which has been proven to effectively improve the performance of other traditional object recognition methods. In this work, we propose PointNet, a new approach inspired by VoxNet and 3D ShapeNets, as an improvement over the existing methods by using density occupancy grids representations for the input data, and integrating them into a supervised Convolutional Neural Network architecture. An extensive experimentation was carried out, using ModelNet - a large-scale 3D CAD models dataset - to train and test the system, to prove that our approach is on par with state-of-the-art methods in terms of accuracy while being able to perform recognition under real-time constraints.

The recognition of group activities using computer vision and pattern recognition methods has been, and still remains, a challenging problem. Most of the research on human behaviour has been focused on recognizing individual issues from actions to behaviours. However, the analysis and recognition of group activities, the relationships of different groups in the scene and the interaction of the individuals in the group is still considered an open problem. This paper proposes a novel representation method to analyse and recognise group activities, called Group Activity Descriptor Vector (GADV). It is calculated from the trajectory described by the group and by the individuals who form it. Specifically, the GADV describes three different components: the trajectory followed by the group, the coherence of the individual trajectories in the group and, finally, the movement relationships among different groups in the scene. The trajectory analysis allows a simple high level understanding of complex groups activities. The GADV representation has been evaluated with different self-organizing neural networks using Behave and Caviar dataset sequences obtaining great accuracy in the recognition of the group activities, outperforming the state of the art methods.

Abstract Schaeffer's sign language consists of a reduced set of gestures designed to help children with autism or cognitive learning disabilities to develop adequate communication skills. Our automatic recognition system for Schaeffer's gesture language uses the information provided by an RGB‐D camera to capture body motion and recognize gestures using dynamic time warping combined with k‐nearest neighbors methods. The learning process is reinforced by the interaction with the proposed system that accelerates learning itself thus helping both children and educators. To demonstrate the validity of the system, a set of qualitative experiments with children were carried out. As a result, a system which is able to recognize a subset of 11 gestures of Schaeffer's sign language online was achieved.

We consider the problem of processing point cloud sequences. In particular, we represent and track objects in dynamic scenes acquired using low-cost sensors such as the Kinect. A neural network based approach is proposed to represent and estimate 3D objects motion. This system addresses multiple computer vision tasks such as object segmentation, representation, motion analysis and tracking. The use of a neural network allows the unsupervised estimation of motion and the representation of objects in the scene. This proposal avoids the problem of finding corresponding features while tracking moving objects. A set of experiments are presented that demonstrate the validity of our method to track 3D objects. Favorable results are presented demonstrating the capabilities of the GNG algorithm for this task.

The object recognition task on 3D scenes is a growing research field that faces some problems relative to the use of 3D point clouds. In this work, we focus on dealing with the noise in the clouds through the use of the Growing Neural Gas (GNG) network filtering algorithm. The GNG method is able to represent the input data with a desired amount of neurons while preserving the topology of the input space. The selected recognition pipeline works describing extracted keypoints of the clouds, grouping and comparing it to detect the presence of an object in the scene, through a hypothesis verification algorithm. Experiments show how the GNG method yields better recognitions results that others filtering algorithms when noise is present.

Nowadays, non-rigid registration problem is an active research topic in computer vision. Various proposals exist which face the problem from different perspectives, but it is still a challenging problem. Currently, with the new low-cost RGB-D sensors, the use of both, color and 3D information, is getting more interest in many applications. In this paper, we present a non-rigid registration technique based on CPD, and including color information along with 3D data, to estimate the non-rigid transformation. As the input data size is critical in the processing time, a sampling technique is required. Five sampling techniques are evaluated: a bilinear sampling, a normal-based, a color-based, a combination of the normal and color-based samplings, and a Growing Neural Gas based approach. All of them have been evaluated with the already presented non-rigid registration methods. Results show the performance of each sampling method, obtaining better results for the registration process using color-based sampling techniques.

Current RGB-D sensors provide a big amount of valuable information for mobile robotics tasks like 3D map reconstruction, but the storage and processing of the incremental data provided by the different sensors through time quickly becomes unmanageable. In this work, we focus on 3D maps representation and we propose the use of a Growing Neural Gas (GNG) network as a 3D representation model of the input data. GNG method is able to represent the input data with a desired amount of neurons while preserving the topology of the input space. Experiments show how GNG method yields better input space adaptation than other state-of-the-art 3D map representation methods.

With the advent of low-cost 3D sensors and 3D printers, surface reconstruction has become an important research topic in the last years. In this work, we propose an automatic method for 3D surface reconstruction from raw unorganized point clouds acquired using low-cost sensors. We have modified the Growing Neural Gas (GNG) network, which is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation, to perform 3D surface reconstruction of different real-world objects. Some improvements have been made on the original algorithm considering colour information during the learning stage and creating complete triangular meshes instead of basic wire-frame representations. The proposed method is able to create 3D faces online, whereas existing 3D reconstruction methods based on Self-Organizing Maps (SOMs) required post-processing steps to close gaps and holes produced during the 3D reconstruction process. Performed experiments validated how the proposed method improves existing techniques removing post-processing steps and including colour information in the final triangular mesh.

3D sensors provide valuable information for mobile robotic tasks like scene classification or object recognition, but these sensors often produce noisy data that makes impossible applying classical keypoint detection and feature extraction techniques. Therefore, noise removal and downsampling have become essential steps in 3D data processing. In this work, we propose the use of a 3D filtering and downsampling technique based on a Growing Neural Gas (GNG) network. GNG method is able to deal with outliers presents in the input data. These features allows to represent 3D spaces, obtaining an induced Delaunay Triangulation of the input space. Experiments show how GNG method yields better input space adaptation to noisy data than other filtering and downsampling methods like Voxel Grid. It is also demonstrated how the state-of-the-art keypoint detectors improve their performance using filtered data with GNG network. Descriptors extracted on improved keypoints perform better matching in robotics applications as 3D scene registration.

The classical connectionist models are not well suited to working with data varying over time. According to this, temporal connectionist models have emerged and constitute a continuously growing research field. In this paper we present a novel supervised recurrent neural network architecture (SARASOM) based on the Associative Self-Organizing Map (A-SOM). The A-SOM is a variant of the Self-Organizing Map (SOM) that develops a representation of its input space as well as learns to associate its activity with an arbitrary number of additional inputs. In this context the A-SOM learns to associate its previous activity with a delay of one iteration. The performance of the SARASOM was evaluated and compared with the Elman network in a number of prediction tasks using sequences of letters (including some experiments with a reduced lexicon of 10 words). The results are very encouraging with SARASOM learning slightly better than the Elman network.

Several recent works deal with 3D data in mobile robotic problems, e.g. mapping. Data come from any kind of sensor (time of flight cameras and 3D lasers) providing a huge amount of unorganized 3D data. In this paper we detail an efficient method to build complete 3D models from a Growing Neural Gas (GNG). We show that the use of GNG provides better results than other approaches. The GNG obtained is then applied to a sequence. From GNG structure, we propose to calculate planar patches and thus obtaining a fast method to compute the movement performed by a mobile robot by means of a 3D models registration algorithm. Final results of 3D mapping are also shown.

In this paper, we present a novel coarse-to-fine visual localization approach: contextual visual localization. This approach relies on three elements: (i) a minimal-complexity classifier for performing fast coarse localization (submap classification); (ii) an optimized saliency detector which exploits the visual statistics of the submap; and (iii) a fast view-matching algorithm which filters initial matchings with a structural criterion. The latter algorithm yields fine localization. Our experiments show that these elements have been successfully integrated for solving the global localization problem. Context, that is, the awareness of being in a particular submap, is defined by a supervised classifier tuned for a minimal set of features. Visual context is exploited both for tuning (optimizing) the saliency detection process, and to select potential matching views in the visual database, close enough to the query view.

Several works deal with 3D data in SLAM problem. Data come from a 3D laser sweeping unit or a stereo camera, both providing a huge amount of data. In this paper, we detail an efficient method to extract planar patches from 3D raw data. Then, we use these patches in an ICP-like method in order to address the SLAM problem. Using ICP with planes is not a trivial task. It needs some adaptation from the original ICP. Some promising results are shown for outdoor environment.

In this paper we propose a method for extracting the planes from a 3D dense map. Three-dimensional data is acquired using active stereo in order to fill texture gaps which are typical in indoor environments. Then, a randomized SLAM algorithm recently proposed by the authors is applied to compute the 3D map by teleoperating a mobile robot. A 3D mesh generation algorithm specially designed to triangulate not solids but open objects, is the basis for computing the main planes in the map after clustering the normals of the vertices in the mesh. Finally, we present our experimental results in indoor environments.

We propose two Bayesian methods for junction classification which evolve from the Kona method: a region-based method and an edge-based method. Our region-based method computes a one-dimensional (1-D) profile where wedges are mapped to intervals with homogeneous intensity. These intervals are found through a growing-and-merging algorithm driven by a greedy rule. On the other hand, our edge-based method computes a different profile which maps wedge limits to peaks of contrast, and these peaks are found through thresholding followed by nonmaximum suppression. Experimental results show that both methods are more robust and efficient than the Kona method, and also that the edge-based method outperforms the region-based one.