As delivery companies continue to explore the use of drones, the need for efficient and safe operation in urban environments becomes increasingly critical. Market-wide versions of drone delivery services will necessarily spread many drones, especially in big cities. In this scenario, avoiding collisions with other drones or typical obstacles in urban spaces is fundamental. This paper proposes and evaluates, via simulation and analytical modeling, an aerial delivery service scenario and three autonomous geometric approaches for collision avoidance. We compare our approaches with three simple methods – DoNothing (not detouring), Random, and aviation-like Rightward – and two state-of-the-art geometric approaches. Simulation experiments consider different fleet sizes with constant and Poisson drone arrival rates and drones randomly choosing one of different altitudes for the cruise flight. Contrary to our expectations, the Random and Rightward approaches increase the collisions compared with DoNothing, making the latter our baseline. Our approaches significantly reduce collisions in all experiments and deal with more drones within the detection radius, showing that collisions are more complex to avoid. Comparing the collision rate, successful trips, and the number of flying drones reveals that the efficiency in avoiding collisions reduces the number of successful trips by increasing the number of active drones. Regardless of the expected reduction in collisions, more altitudes do not eliminate them. These results indicate the need for more sophisticated approaches to reduce or eliminate collisions. The analytical modeling using Markov Chains corroborates the simulation results by shedding some light on and helping explain the simulation results.

The development of autonomous agents that perform tasks with the same dexterity as performed by humans is one of the challenges of artificial intelligence and robotics. This motivates the research on intelligent agents, since the agent must choose the best action in a dynamic environment in order to maximise the final score. In this context, the present paper introduces a novel algorithm for Qualitative Case-Based Reasoning and Learning (QCBRL), which is a case-based reasoning system that uses qualitative spatial representations to retrieve and reuse cases by means of relations between objects in the environment. Combined with reinforcement learning, QCBRL allows the agent to learn new qualitative cases at runtime, without assuming a pre-processing step. In order to avoid cases that do not lead to the maximum performance, QCBRL executes case-base maintenance, excluding these cases and obtaining new (more suitable) ones. Experimental evaluation of QCBRL was conducted in a simulated robot-soccer environment, in a real humanoid-robot environment and on simple tasks in two distinct gridworld domains. Results show that QCBRL outperforms traditional RL methods. As a result of running QCBRL in autonomous soccer matches, the robots performed a higher average number of goals than those obtained when using pure numerical models. In the gridworlds considered, the agent was able to learn optimal and safety policies.

A major challenge in gene regulatory networks (GRN) of biological systems is to discover when and what interventions should be applied to shift them to healthy phenotypes. A set of gene activity profiles, called basin of attraction (BOA), takes this network to a specific phenotype; therefore, a healthy BOA leads the GRN to a healthy phenotype. However, without the complete observability of the genes, it is not possible to identify whether the current BOA is healthy. In this article we investigate external interventions in GRN with partial observability aiming to bring it to healthy BOAs. We propose a new batch reinforcement learning method (BRL), called mSFQI, to define intervention strategies based on the probabilities of the gene activity profiles being in healthy BOAs, which are calculated from a set of previous observed experiences. BRL uses approximation functions and repeated applications of previous experiences to accelerate learning. Results demonstrate that our proposal can quickly shift a partially observable GRN to healthy BOAs, while reducing the number of interventions. In addition, when observability is poor, mSFQI produces better results when the probabilities for a greater amount of previous observations are available.

Recommender systems have been widely used by large companies in the e-commerce segment as aid tools in the search for relevant contents according to the user’s particular preferences. A wide variety of algorithms have been proposed in the literature aiming at improving the process of generating recom- mendations; in particular, a collaborative, diffusion-based hybrid algorithm has been proposed in the lit- erature to solve the problem of sparse data, which affects the quality of recommendations. This algorithm was the basis for several others that effectively solved the sparse data problem. However, this family of algorithms does not differentiate users according to their profiles. In this paper, a new algorithm is pro- posed for learning the user profile and, consequently, generating personalised recommendations through diffusion, combining novelty with the popularity of items. Experiments performed in well-known datasets show that the results of the proposed algorithm outperform those from both diffusion-based hybrid al- gorithm and traditional collaborative filtering algorithm, in the same settings.

Having the ability to solve increasingly complex problems using Reinforcement Learning (RL) has prompted researchers to start developing a greater interest in systematic approaches to retain and reuse knowledge over a variety of tasks. With Case-based Reasoning (CBR) there exists a general methodology that provides a framework for knowledge transfer which has been underrepresented in the RL literature so far. We formulate a terminology for the CBR framework targeted towards RL researchers with the goal of facilitating communication between the respective research communities. Based on this framework, we propose the Deep Case-based Policy Inference (DECAF) algorithm to accelerate learning by building a library of cases and reusing them if they are similar to a new task when training a new policy. DECAF guides the training by dynamically selecting and blending policies according to their usefulness for the current target task, reusing previously learned policies for a more effective exploration but still enabling the adaptation to particularities of the new task. We show an empirical evaluation in the Atari game playing domain depicting the benefits of our algorithm with regards to sample efficiency, robustness against negative transfer, and performance increase when compared to state-of-the-art methods.

This paper presents an Artificial Immune System framework for solving Multiobjective Optimization Problems that makes use of two immunologic operators with non-immunologic features, called Hybrid Operators, which were inspired from Artificial Neural Networks and Genetic Algorithms techniques. The first of these operators makes use of the concept of Momentum to improve the performance of mutation while the second incorporates the recombination operator. The new AIS framework is called MOHAIS, for Multiobjective Optimization Hybrid Artificial Immune System. The proposed framework was used to implement one traditional AIS algorithm and a new hybrid AIS, based on proposed hybrid operators. Experiments were made to evaluate hybrid operators performance, to compare the hybrid AIS with one traditional AIS and three Multiobjective Optimization Evolutionary Algorithms, in nine traditional benchmark problems. Results showed that hybrid operators improved the performance of the AIS with satisfactory results in all nine scenarios.

This paper presents a novel class of algorithms, called Heuristically-Accelerated Multiagent Reinforcement Learning (HAMRL), which allows the use of heuristics to speed up well-known multiagent reinforcement learning (RL) algorithms such as the Minimax-Q. Such HAMRL algorithms are characterized by a heuristic function, which suggests the selection of particular actions over others. This function represents an initial action selection policy, which can be handcrafted, extracted from previous experience in distinct domains, or learnt from observation. To validate the proposal, a thorough theoretical analysis proving the convergence of four algorithms from the HAMRL class (HAMMQ, HAMQ(λ), HAMQS, and HAMS) is presented. In addition, a comprehensive systematical evaluation was conducted in two distinct adversarial domains. The results show that even the most straightforward heuristics can produce virtually optimal action selection policies in much fewer episodes, significantly improving the performance of the HAMRL over vanilla RL algorithms.

Selecting which algorithms should be used by a mobile robot computer vision system is a decision that is usually made a priori by the system developer, based on past experience and intuition, not systematically taking into account information that can be found in the images and in the visual process itself to learn which algorithm should be used, in execution time. This paper presents a method that uses Reinforcement Learning to decide which algorithm should be used to recognize objects seen by a mobile robot in an indoor environment, based on simple attributes extracted on-line from the images, such as mean intensity and intensity deviation. Two state-of-the-art object recognition algorithms can be selected: the constellation method proposed by Lowe together with its interest point detector and descriptor, the Scale-Invariant Feature Transform and a bag of features approach. A set of empirical evaluations was conducted using a household mobile robots image database, and results obtained shows that the approach adopted here is very promising.