In robotics, object detection in images or videos, obtained in real-time from sensors of robots can be used to support the implementation of service robot tasks (e.g., navigation, model its social behavior, recognize objects in a specific domain), usually accomplished in indoor environments. However, traditional deep learning based object detection techniques present limitations in such indoor environments, specifically related to the detection of small objects and the management of high density of multiple objects. Coupled with these limitations, for specific domains (e.g., hospitals, museums), it is important that the robot, apart from detecting objects, extracts and knows information of the targeted objects. Ontologies, as a part of the Semantic Web, are presented as a feasible option to formally represent the information related to the objects of a particular domain. In this context, this work proposes an object detection and recognition process based on a Deep Learning algorithm, object descriptors, and an ontology. ODROM, an Object Detection and Recognition algorithm supported by Ontologies and applied to Museums, is an implementation to validate the proposal. Experiments show that the usage of ontologies is a good way of desambiguating the detection, obtained with a and$\mathbf{mAP}{@}0.5=0.88$and a$\mathbf{mAP}{@}[0.5:0.95]=61\%$.

Human emotion recognition from visual expressions is an important research area in computer vision and machine learning owing to its significant scientific and commercial potential. Since visual expressions can be captured from different modalities (e.g., face expressions, body posture, hands pose), multi-modal methods are becoming popular for analyzing human reactions. In contexts in which human emotion detection is performed to associate emotions to certain events or objects to support decision making or for further analysis, it is useful to keep this information in semantic repositories, which offers a wide range of possibilities for implementing smart applications. We propose a multi-modal method for human emotion recognition and an ontology-based approach to store the classification results in EMONTO, an extensible ontology to model emotions. The multi-modal method analyzes facial expressions, body gestures, and features from the body and the environment to determine an emotional state; this processes each modality with a specialized deep learning model and applying a fusion method. Our fusion method, called EmbraceNet+, consists of a branched architecture that integrates the EmbraceNet fusion method with other ones. We experimentally evaluate our multi-modal method on an adaptation of the EMOTIC dataset. Results show that our method outperforms the single-modal methods.

With the robotics development, social robots interact with people, demanding they model the human being behavior to increase social navigation, considering proxemic spaces. However, human proxemic preferences can change in function of different social restrictions (e.g., culture, gender, local, the environment). Thus, robots should consider all these aspects to tailor their navigation. Towards an adaptable social navigation, in this article we develop the GProxemic Navigation system that allows identifying the robot localization in a geo-referenced map, with semantic annotations related to social restrictions, in function of which they chose the correct proxemic spaces they most respect in their autonomous navigation process. Results show that the GProxemic Navigation system efficiently obeys the proxemic space through the semantic annotations received.

The present work demonstrates how an autonomous navigation system of `End-to-End' deep learning principles is directly improved in its response process, depending on the information obtained by different input images configurations. For this, a methodology was developed to allow working with RGB and depth images, which were obtained through a Microsoft Kinect V2 sensor device. Three cameras were used for this experiment. The images of the different cameras were concatenated or grouped, generating new and different input configurations from the vision system. To develop the presented methodology, two support and validation systems were implemented. Through the process of computer simulation, it was able to test the first approaches and define the most important ones. In order to validate the proposed methodology and solutions in real world situations, a 1/4 scale automotive vehicle was prototyped. Finally, the experiments shows the importance of the use of multi-cameras systems for a better performance of autonomous navigation systems based on End-to-End learning approach, heaving an average error of 2.41 degrees in the best configuration tested, with three RGB cameras.