Deploying low-altitude unmanned aerial vehicles (UAVs) as aerial base stations (ABSs) is a promising approach to enhance cellular network coverage in dynamic and infrastructure-constrained environments. However, real-world UAV deployment involves complex challenges, including managing partially observable and mobile users, maintaining multi-hop backhaul connectivity to a fixed ground base station, mitigating co-channel interference via minimum UAV separation, and optimizing coverage efficiency, fairness, and energy consumption. This paper proposes a multi-agent deep reinforcement learning (MARL) framework that formulates the dynamic UAV deployment task as a centralized multi-input multi-output policy. Our method employs sequential action masking to guarantee multi-hop connectivity and enforce minimum distance constraints during UAV placement. The policy network integrates graph neural networks, residual convolutional layers, and feature-wise linear modulation to effectively capture spatial relations, inter-UAV interactions, and global and local network states. We validate our approach through extensive simulations across various user density scenarios, demonstrating significant improvements in coverage efficiency, fairness of coverage distribution, and flight energy reduction compared to strong baseline algorithms. Our proposed framework provides a comprehensive solution that simultaneously addresses dynamic user mobility, multi-hop connectivity, interference mitigation, and energy efficiency in UAV-assisted cellular networks.

This work proposes a scheme that allows learning complex multi-agent behaviors in a sample efficient manner, applied to 2v2 soccer. The problem is formulated as a Markov game, and solved using deep reinforcement learning. We propose a basic multi-agent extension of TD3 for learning the policy of each player, in a decentralized manner. To ease learning, the task of 2v2 soccer is divided in three stages: 1v0, 1v1 and 2v2. The process of learning in multi-agent stages (1v1 and 2v2) uses agents trained in a previous stage as fixed opponents. In addition, we propose using experience sharing, a method that shares experience from a fixed opponent, trained in a previous stage, for training the agent currently learning, and a form of frame-skipping, to raise performance significantly. Our results show that high quality soccer play can be obtained with our approach in just under 40M interactions. A summarized video of the resulting game play can be found in https://youtu.be/pScrKNqfELE.

In order to address the simulation-to-reality-gap, in this paper a methodology for the real-time generation of realistic images in robotic simulation environments is proposed. The images rendered by the simulator are first segmented, and then a generative neural network transforms them into realistic images. This allows training object recognition methods in situations dynamically generated by the simulator, but using realistic images. The generative neural network is trained using a database obtained using an instance segmentation network (Mask R-CNN). The whole methodology is validated in the soccer robotics domain. The reported experiments show that CNN based object detectors trained in simulation, using the generated realistic images, can be directly transferred to reality, where state-of-the art results are obtained. Moreover, we show how the training process of these detectors is fast, easy, and does not require the repetitive use of robots, which is time consuming for humans.

Deep Reinforcement Learning (DRL) has become a powerful methodology to solve complex decision-making problems. However, DRL has several limitations when used in real-world problems (e.g., robotics applications). For instance, long training times are required and cannot be accelerated in contrast to simulated environments, and reward functions may be hard to specify/model and/or to compute. Moreover, the transfer of policies learned in a simulator to the real-world has limitations (reality gap). On the other hand, machine learning methods that rely on the transfer of human knowledge to an agent have shown to be time efficient for obtaining well performing policies and do not require a reward function. In this context, we analyze the use of human corrective feedback during task execution to learn policies with high-dimensional state spaces, by using the D-COACH framework, and we propose new variants of this framework. D-COACH is a Deep Learning based extension of COACH (COrrective Advice Communicated by Humans), where humans are able to shape policies through corrective advice. The enhanced version of DCOACH, which is proposed in this paper, largely reduces the time and effort of a human for training a policy. Experimental results validate the efficiency of the D-COACH framework in three different problems (simulated and with real robots), and show that its enhanced version reduces the human training effort considerably, and makes it feasible to learn policies within periods of time in which a DRL agent do not reach any improvement.

We propose a Visual-SLAM based localization and navigation system for service robots. Our system is built on top of the ORB-SLAM monocular system but extended by the inclusion of wheel odometry in the estimation procedures. As a case study, the proposed system is validated using the Pepper robot, whose short-range LIDARs and RGB-D camera do not allow the robot to self-localize in large environments. The localization system is tested in navigation tasks using Pepper in two different environments: a medium-size laboratory, and a large-size hall.

The RoboCup@Home league has been founded in 2006with the idea to drive research in AI and related fieldstowards autonomous and interactive robots that copewith real life tasks in supporting humans in everday life.The yearly competition format establishes benchmarkingas a continuous process with yearly changes insteadof a single challenge. We discuss the current state andfuture perspectives of this endeavor.

The main goal of this paper is to present a simple, but robust algorithm for detecting white goals in the context of the RoboCup SPL (Standard Platform League). White goals will be used for the first time in the SPL competitions in 2015. The main features of the algorithm are a robust search strategy for detecting the goal posts, and the use of the Y channel image, instead of the color segments, for determining and characterizing the goal posts and the horizontal crossbar. This last aspect is crucial for detecting a white goal placed in a white background. The algorithm is validated in the real world (real robot in a SPL field), showing its ability to detect the while goals even when they are observed in a white background.

An interactive learning framework that allows non-expert humans to shape a policy through corrective advice, using a binary signal in the action domain of the robot/agent, is proposed. One of the most innovative features of COACH (COrrective Advice Communicated by Humans), the proposed framework, is a mechanism for adaptively adjusting the amount of human feedback that a given action receives, taking into consideration past feedback. The performance of COACH is compared with the one of TAMER (Teaching an Agent Manually via Evaluative Reinforcement), ACTAMER (Actor-Critic TAMER), and an autonomous agent trained using SARSA(λ) in two reinforcement learning problems: ball dribbling and Cart-Pole balancing. COACH outperforms the other learning frameworks in the reported experiments. In addition, results show that COACH is able to transfer successfully human knowledge to agents with continuous actions, being a complementary approach to TAMER, which is appropriate for teaching in discrete action domains.

Hierarchical task decomposition strategies allow robots and agents in general to address complex decision-making tasks. Layered learning is a hierarchical machine learning paradigm where a complex behavior is learned from a series of incrementally trained sub-tasks. This paper describes how layered learning can be applied to design individual behaviors in the context of soccer robotics. Three different layered learning strategies are implemented and analyzed using a ball-dribbling behavior as a case study. Performance indices for evaluating dribbling speed and ball-control are defined and measured. Experimental results validate the usefulness of the implemented layered learning strategies showing a trade-off between performance and learning speed.

This paper presents an efficient active vision system which controls the head of a humanoid soccer robot. The system explicitly separates static information obtained offline from the map, and dynamic information from mobile objects, such as the ball and other players. Both types of information are mapped and handled in a simplified structure called action space , which assigns scores to each possible action of the robot’s head. Scores also consider the movement constraints of the robot’s head. Due to its simplicity and efficient information handling, the proposed active vision system is able to run in real-time in less than 1 ms. The performance of the system in a robot soccer environment is tested via simulation and real experiments.

The main goal of this paper is to propose a framework for providing an episodic long-term memory for a robot, which includes methods for acquiring, storing, updating, managing and using episodic information. This will give a robot the ability to incorporate past experiences when interacting with humans, so that the data that the robot learns transcends each session, and thus gives continuity to its activities and behaviors. As a proof of concept, the implementation of an episodic long-term memory for the Bender robot is described. This includes the implementation and evaluation of a behavior called Conversation , which allows Bender to interact with people using the information stored in the episodic memory.

Scientific competitions are becoming more common in many research areas of artificial intelligence and robotics, since they provide a shared testbed for comparing different solutions and enable the exchange of research results. Moreover, they are interesting for general audiences and industries. Currently, many major research areas in artificial intelligence and robotics are organizing multiple-year competitions that are typically associated with scientific conferences. One important aspect of such competitions is that they are organized for many years. This introduces a temporal evolution that is interesting to analyze. However, the problem of evaluating a competition over many years remains unaddressed. We believe that this issue is critical to properly fuel changes over the years and measure the results of these decisions. Therefore, this article focuses on the analysis and the results of evolving competitions. In this article, we present the [email protected] competition, which is the largest worldwide competition for domestic service robots, and evaluate its progress over the past seven years. We show how the definition of a proper scoring system allows for desired functionalities to be related to tasks and how the resulting analysis fuels subsequent changes to achieve general and robust solutions implemented by the teams. Our results show not only the steadily increasing complexity of the tasks that [email protected] robots can solve but also the increased performance for all of the functionalities addressed in the competition. We believe that the methodology used in [email protected] for evaluating competition advances and for stimulating changes can be applied and extended to other robotic competitions as well as to multi-year research projects involving Artificial Intelligence and Robotics.

Circular data is very relevant in many fields such as Geostatistics, Mobile Robotics and Pose Estimation. However, some existing angular regression methods do not cope with arbitrary nonlinear functions properly. Moreover, some other regression methods that do cope with nonlinear functions, like Gaussian Processes, are not designed to work well with angular responses. This paper presents two novel methods for circular regression based on Gaussian Processes. The proposed methods were tested on both synthetic data from basic functions, and real data obtained from a computer vision application. In these experiments, both proposed methods showed superior performance to that of Gaussian Processes.

Early detection of fall risk can reduce health costs associated with surgery, rehabilitation, imaging studies, hospitalizations, and medical evaluations. This paper proposes a measurement-focused study oriented to evaluate a new methodology for assessing fall risk using low-cost and off-the-shelf devices. The proposed methodology consists of a data acquisition system, a data analysis system, and a fall risk assessment system. The data acquisition system is composed by a standard notebook computer and video game input devices: a Kinect, a Wii balance board, and two Wii motion controllers. The data analysis system and the fall risk assessment system, in turn, use signal processing, data mining, and computational intelligence methods, in order to analyze the acquired data for determining the fall risk of the subject under analysis. This methodology includes six static and two dynamic tests. Experiments were conducted on a population of 37 subjects: 16 with falling background, and 21 with nonfalling background. These two groups have the same age distribution. As nonlinear binary classification techniques were used, methodologies based on confidence intervals are not applicable and then tenfold cross validation was used to estimate accuracy. Hence, such a methodology can classify the fall risk as high or low, with an accuracy of 89.2%. The proposed methodology allows the construction of low-cost, portable, replicable, objective, and reliable fall risk assessment systems.

The main goal of this article is to describe Bender, a general-purpose social robot with human--robot interaction capabilities, and to report and analyze its applicability in three different natural environments: (i) home settings, (ii) school classrooms, and (iii) public places. Evaluation of the robot's performance is based on its ability to detect and identify humans in those settings, its ability to express emotions, and its ability to interact naturally with humans. The obtained results, although preliminary, indicate that the robot has great acceptance from expert and non-expert human users alike, and that it is able to interact successfully with humans using human-like interaction mechanisms, such as speech and visual cues (particularly facial information). It is remarkable that the robot even has the potential to teach children in a real classroom setting. Further experiments will be conducted in order to better assess the robot's potential.

Building robust and fast object detection systems is an important goal of computer vision. A problem arises when several object types are to be detected, because the computational burden of running several specific classifiers in parallel becomes a problem. In addition the accuracy and the training time can be greatly affected. Seeking to provide a solution to these problems, we extend cascade classifiers to the multiclass case by proposing the use of multiclass coarse-to-fine (CTF) nested cascades. The presented results show that the proposed system scales well with the number of classes, both at training and running time.

The appropriate management of fall situations - i.e. fast instability detection, avoidance of unintentional falls, falling without damaging the body, fast recovering of the standing position after a fall - is an essential ability of biped humanoid robots. This issue is especially important for humanoid robots carrying out demanding movements such as walking in irregular surfaces, running or practicing a given sport (e.g. soccer). In a former contribution we have addressed the design of low-damage fall sequences, which can be activated/triggered by the robot in case of a detected unintentional fall or an intentional fall (common situation in robot soccer). In this article we tackle the detection of instability and the avoidance of falls in biped humanoids, as well as the integration of all components in a single framework. In this framework a fall can be avoided or a falling sequence can be triggered depending on the detected instability's degree. The proposed fall detection and fall avoidance subsystems are validated in real world-experiments with biped humanoid robots.

In this paper we illustrate how self organizing maps (SOM) can be used to assess the influence of natural and anthropogenic factors in an impacted fluvial system. We use the Growing Cell Structures (GCS) algorithm to study a multivariable data set that includes the chemical composition of 90 stream sediments, collected along the Rapel Fluvial System, in Central Chile. The GCS algorithm, an extension of SOM, allows simultaneous adaptation of the position of the map pointers in the input space and the topology of the output space. Therefore it helps to find visual representations of geochemical data that simplify the analysis and yet allow all relevant information to be preserved. The results show that the data can be separated into a limited number of groups with clear and different chemical characteristics. Each group corresponds to stream sediments that are restricted to a specific portion of the fluvial system. In the upper part of the catchment, the Cachapoal River is characterized by elevated Cu-Mo-As-Sb concentrations that reflect input of mining-derived material, coupled with high B values associated with agro-industrial activities. In turn, the high P concentrations of the Tinguiririca River reflect an extensive use of phosphates in its floodplain. Due the influence of the Rapel Lake, the lower part of the catchment is mainly controlled by natural factors such as downstream changes in the lithologycal composition of the bedrock and in the hydrodynamic conditions of the Rapel River. Comparisons are made between the results obtained by using GCS and SOM. Both techniques enable the recognition of the cluster structure and their chemical characteristics. However, GCS requires a simpler map structure with less number of units, which optimizes translating the relevant information to geographical maps and make associations with spatially distributed features such as bedrock geology, urbanization, industrial activities and land use.

In this article, a virtual environment for realistic testing and training of face detection and recognition systems under uncontrolled conditions is proposed. The key elements of this tool are a simulator and real face and background images taken under real-world conditions with different acquisition conditions. Inside the simulated environment, an observing agent, the one with the ability to recognize faces, can navigate and observe the real face images, at different distances, angles and with indoor or outdoor illumination. During the face recognition process, the agent can actively change its viewpoint and relative distance to the faces in order to improve the recognition results. The virtual environment provides all behaviors to the agent (navigation, positioning, face's image composing under different angles, etc.), except the ones related with the recognition of faces. This tool could be of high interest in HRI applications related with the visual recognition of humans. It allows comparing and quantifying the face recognition capabilities of service robots, and in general intelligent machines, under exactly equal working conditions. The applicability of the proposed tool is validated in the comparison of state of the art face detection and recognition methods.

In this contribution we apply a self organizing map algorithm to And groups or clusters in two different multivariable geochemical datasets. The first dataset includes whole rock chemical data of four different island arc volcanic rock types: basalts, andesites, dacites and rhyolites. The second dataset includes compositional data of clinopyroxenes from sixteen different volcanic rock types of mainly mafic composition. The outcomes of the method clearly reveal the cluster structure of both datasets. The relevant factors responsible for the cluster structures can also be visualized and meaningful correlations between relevant discriminating factors identified. For the island arc dataset the results show that there are systematic changes of major element concentrations between the different island arc volcanic rock types. These chemical changes resemble the differentiation trend from basalts to rhyolites. The results also show that for this dataset the cluster structure can be improved by using the log-ratio values rather than the not normalized major element data. The results from the analysis of the pyroxene dataset show that the composition of clinopyroxenes clearly reflects the chemical differences and similarities that exist between different basaltic magma types.

This paper presents the application of evolutionary multi-objective optimization (EMO) to the improvement of a face detection system. The face detection system is based on the boosted cascade system, and analyzes image positions on different scales in a three-step-procedure. Based on threshold settings, the algorithm decides whether to continue with the test on a finer scale at the current position. Thus, the thresholds for all scales and stages have a major influence on the performance of the system, and become the subject of the evolutionary optimization according to three objectives: low false positive rate, high detection rate and low processing time. The used EMO is the extension of the standard genetic algorithm to the EMO case by using fuzzy Pareto dominance as a meta-heuristic. The application of this EMO to the face detection system is explored and discussed using images from a standard face detection benchmark dataset. From the runtime analysis of the EMO it can be concluded that the algorithm reliably approaches the Pareto set of the problem.

Eigenspace-based face recognition corresponds to one of the most successful methodologies for the computational recognition of faces in digital images. Starting with the Eigenface-Algorithm, different eigenspace-based approaches for the recognition of faces have been proposed. They differ mostly in the kind of projection method used (standard, differential, or kernel eigenspace), in the projection algorithm employed, in the use of simple or differential images before/after projection, and in the similarity matching criterion or classification method employed. The aim of this paper is to present an independent comparative study among some of the main eigenspace-based approaches. We believe that carrying out independent studies is relevant, since comparisons are normally performed using the implementations of the research groups that have proposed each method, which does not consider completely equal working conditions for the algorithms. Very often, a contest between the abilities of the research groups rather than a comparison between methods is performed. This study considers theoretical aspects as well as simulations performed using the Yale Face Database, a database with few classes and several images per class, and FERET, a database with many classes and few images per class.

Skin detection or segmentation is employed in many tasks related to the detection and tracking of humans and human-body parts. However, skin detection is not robust enough for dealing with some real-world conditions, such as changing lighting conditions and complex backgrounds containing surfaces and objects with skin-like colors. This situation can be improved by incorporating context information in the skin detection process. For this reason, a skin detection approach that uses neighborhood information is proposed. This idea is implemented through a diffusion process that allows a robust segmentation of skin regions at a high processing speed.

Different eigenspace-based approaches have been proposed for the recognition of faces. They differ mostly in the kind of projection method being used and in the similarity matching criterion employed. The aim of this paper is to present a comparative study between some of these different approaches. This study considers theoretical aspects as well as experiments performed using a face database with a few number of classes (Yale) and also with a large number of classes (FERET).

An essential task in almost any pattern recognition system is the extraction of feature vectors, which are then used to perform a classification. Depending on the context of this classification, these feature vectors are expected to present invariance under basic transformations such as translation, scaling or rotation. Thus, every problem needs a careful selection of feature variables, which so far is mostly done by hand. Neural networks have been used successfully as classifiers for a long time, but only recently they have begun to be employed for automatic selection of feature variables. The ASSOM, ASGCS and ASGFC neural models are able to automatically select features variables (filters) for the segmentation of textures. In the paper three different texture segmentation architectures TEXSOM, TEXGFC and TEXSGFC, which are based on the mentioned neural models, are described.

The simultaneous progress in sensor and signal processing technologies stimulate the implementation of more refined pattern recognition systems in order to solve problems of increasing complexity. This progress induced the development of the here presented framework for the fusion of chromatic, infrared and textural information. The framework is based on different aspects of the processing of visual information in the human brain. Some organizational principles of the multisensorial information fusion in higher associative areas are also reflected on it. Preliminary results, realized on a simplified version of the framework, show the validity of the bio-inspired approach on the resolution of multisensorial image fusion.

The automatic selection of feature variables is a task of increasing interest in the field of pattern recognition. Neural models have recently been used for this purpose. Among other models, the adaptive-subspace SOM (ASSOM) stands out because of its simplicity and biological plausibility. However, the main drawback of its application in image processing systems is that a priori information is necessary to choose a suitable network size and topology in advance. This article introduces the adaptive-subspace growing cell structures (ASGCS) network, which corresponds to a further improvement of the ASSOM that overcomes its main drawbacks. The ASGCS network is described and some examples of automatic generation of Gabor-like feature filter are given.