Communication constraints can significantly impact robots’ ability to share information, coordinate their movements, and synchronize their actions, thus limiting coordination in Multi-Robot Exploration (MRE) applications. In this work, we address these challenges by modeling the MRE application as a DEC-POMDP and designing a joint policy that follows a rendezvous plan. This policy allows robots to explore unknown environments while intermittently sharing maps opportunistically or at rendezvous locations without being constrained by joint path optimizations. To generate the rendezvous plan, robots represent the MRE task as an instance of the Job Shop Scheduling Problem (JSSP) and minimize JSSP metrics. They aim to reduce waiting times and increase connectivity, which correlates to the DEC-POMDP rewards and time to complete the task. Our simulation results suggest that our method is more efficient than using relays or maintaining intermittent communication with a base station, being a suitable approach for Multi-Robot Exploration. We developed a proof-of-concept using the Robot Operating System (ROS) that is available at: https://github.com/multirobotplayground/Noetic-Multi-Robot-Sandbox.

Asset creation in game development usually requires multiple iterations until a final version is achieved. This iterative process becomes more significant when the content is pixel art, in which the artist carefully places each pixel. We hypothesize that the problem of generating character sprites in a target pose (e.g., facing right) given a source (e.g., facing front) can be framed as an image-to-image translation task. Then, we present an architecture of deep generative models that takes as input an image of a character in one domain (pose) and transfers it to another. We approach the problem using generative adversarial networks (GANs) and build on Pix2Pix’s architecture while leveraging some specific characteristics of the pixel art style. We evaluated the trained models using four small datasets (less than 1k) and a more extensive and diverse one (12k). The models yielded promising results, and their generalization capacity varies according to the dataset size and variability. After training models to generate images among four domains (i.e., front, right, back, left), we present an early version of a mixed-initiative sprite editor that allows users to interact with them and iterate in creating character sprites.

In collectible card games, developers face the challenge of creating new, and interesting cards that are not too strong or game-breaking, retaining the game’s overall balance. Over time, this becomes challenging due to the sheer volume of the published content. In this article, we propose a framework for generating models capable of recommending resource scales, a pivotal point in balancing. We evaluate the usage of several state-of-the-art neural architectures to learn representations for text followed by gradient boosting decision trees to incorporate remaining features. Throughout our analysis, we present various explanation tools that should empower game developers, and aid them with new insights. In particular, we present the sets of words that drive the model in diverse situations, such as when it was inaccurate by a small margin. We also exhibit instances where textual features cannot give an accurate prediction, requiring additional information. Our method achieves a mean reciprocal rank of$+0.8$when evaluated on popular card games, even though superficially identical cards might have distinct costs.

This paper presents a novel approach that allows a swarm of heterogeneous robots to produce simultaneously segregative and flocking behaviors using only local sensing. These behaviors have been widely studied in swarm robotics and their combination allows the execution of several complex tasks. Our approach consists of modeling the swarm as a Gibbs Random Field (GRF) and using appropriate potential functions to reach segregation, cohesion and consensus on the velocity of the swarm. Simulations and proof-of-concept experiments using real robots are presented to evaluate the performance of our methodology in comparison to some of the state-of-the-art works that tackle segregative behaviors.

This paper presents a novel methodology that allows a swarm of robots to perform a cooperative transportation task. Our approach consists of modeling the swarm as a Gibbs Random Field (GRF), taking advantage of this framework’s locality properties. By setting appropriate potential functions, robots can dynamically navigate, form groups, and perform co- operative transportation in a completely decentralized fashion. Moreover, these behaviors emerge from the local interactions without the need for explicit communication or coordination. To evaluate our methodology, we perform a series of simulations and proof-of-concept experiments in different scenarios. Our results show that the method is scalable, adaptable, and robust to failures and changes in the environment.

We propose a way to model and integrate HPC scheduling simulators into a popular Reinforcement Learning toolkit. We show experimentally that such an approach not only aids researchers being able to iterate faster by means of software reuse, but also to achieve state-of-the-art performance with 10x less interactions with the environment. We validate the simulation model's correctness by using unit tests, assertions and experimental comparisons. We also share an open source implementation of the model that will benefit researchers in resource management tasks assisted by Machine Learning.

Algorithm selection-the mapping of problem instances to algorithms-has been successfully applied to a variety of complex theoretical and practical problems, including computer games. In this paper, we extend the traditional framework, which considers a single decision maker, to adversarial settings, by modeling algorithm selection as a normal-form game. In this “game of algorithm selection,” agents select algorithms to play a computer game on their behalf. The game's payoff matrix stores the relative performance among algorithms. We also consider nonstationary scenarios, where algorithms can learn from previous matches. We apply this approach to real-time strategy game StarCraft, using bots developed for the game as our algorithms. Our experiments suggest that minimax-Q is a suitable method for algorithm selection in both stationary and nonstationary conditions. We proceed by implementing our approach in MegaBot, a fully capable StarCraft bot. MegaBot showed robustness to nonstationarity, competing in the difficult scenario of StarCraft AI tournaments. MegaBot successfully learns how to select algorithms, exhibiting increasing win rates with tournament progress. In 2016, its debut year, it left 60% of opponents behind in two out of three tournaments. In 2017, however, MegaBot faced difficulties as its algorithm portfolio got outdated compared to newer entries.

Large state and action spaces are very challenging to reinforcement learning. However, in many domains there is a set of algorithms available, which estimate the best action given a state. Hence, agents can either directly learn a performance-maximizing mapping from states to actions, or from states to algorithms. We investigate several aspects of this dilemma, showing sufficient conditions for learning over algorithms to outperform over actions for a finite number of training iterations. We present synthetic experiments to further study such systems. Finally, we propose a function approximation approach, demonstrating the effectiveness of learning over algorithms in real-time strategy games.

Corpora used to learn open-domain Question-Answering (QA) models are typically collected from a wide variety of topics or domains. Since QA requires understanding natural language, open-domain QA models generally need very large training corpora. A simple way to alleviate data demand is to restrict the domain covered by the QA model, leading thus to domain-specific QA models. While learning improved QA models for a specific domain is still challenging due to the lack of sufficient training data in the topic of interest, additional training data can be obtained from related topic domains. Thus, instead of learning a single open-domain QA model, we investigate domain adaptation approaches in order to create multiple improved domain-specific QA models. We demonstrate that this can be achieved by stratifying the source dataset, without the need of searching for complementary data unlike many other domain adaptation approaches. We propose a deep architecture that jointly exploits convolutional and recurrent networks for learning domain-specific features while transferring domain-shared features. That is, we use transferable features to enable model adaptation from multiple source domains. We consider different transference approaches designed to learn span-level and sentence-level QA models. We found that domain-adaptation greatly improves sentence-level QA performance, and span-level QA benefits from sentence information. Finally, we also show that a simple clustering algorithm may be employed when the topic domains are unknown and the resulting loss in accuracy is negligible.

In fighting games, players can perform many different actions at each instant of time, leading to an exponential number of possible sequences of actions. Some of these combinations can lead to unexpected behaviors, which can compromise the game design. One example of these unexpected behaviors is the occurrence of long or infinite combos, a long sequence of actions that does not allow any reactions from the opponent. Finding these sequences is essential to ensure fairness in fighting games, but evaluating all possible sequences is a time consuming task. In this paper, we propose the use of an evolutionary algorithm to find combos on a fighting game. The main idea is to use a genetic algorithm to evolve a population composed of sequences of inputs and, using an adequate fitness function, select the ones that are more suitable to be considered combos. We performed a series of experiments and the results show that the proposed approach was not only successful in finding combos, managing to find unexpected sequences, but also superior to previous methods.

Several natural systems adopt self-sorting mechanisms based on segregative behaviors. Among these, cell segregation is of particular interest since it plays an important role in the formation of tissues, organs, and living organisms. The Differential Adhesion Hypothesis states that cells naturally segregate because of differences in affinity, which lead similar cells to strongly adhere to each other. By exploring this principle, we propose a controller that can segregate a heterogeneous swarm of robots according to the characteristics of each agent, such that similar robots form homogeneous teams and dissimilar robots are segregated. We apply LaSalle's Invariance Principle to show convergence and perform simulated experiments in order to demonstrate the robustness and effectiveness of the proposed controller. Results show that our approach allows a swarm of multiple heterogeneous robots to segregate in a coherent and smooth fashion, without any inter-agent collisions.

The focus of this study is on the design of feedback control laws for swarms of robots that are based on models from fluid dynamics. We apply an incompressible fluid model to solve a pattern generation task. Possible applications of an efficient solution to this task are surveillance and the cordoning off of hazardous areas. More specifically, we use the smoothed-particle hydrodynamics (SPH) technique to devise decentralized controllers that force the robots to behave in a similar manner to fluid particles. Our approach deals with static and dynamic obstacles. Considerations such as finite size and nonholonomic constraints are also addressed. In the absence of obstacles, we prove the stability and convergence of controllers that are based on the SPH method. Computer simulations and actual robot experiments are shown to validate the proposed approach.

Safe and efficient navigation of robotic swarms is an important research problem. One of the main challenges in this area is to avoid congestion, which usually happens when large groups of robots share the same environment. In this paper, we propose the use of hierarchical abstractions in conjunction with simple traffic control rules based on virtual forces to avoid congestion in swarm navigation. We perform simulated and real experiments in order to study the feasibility and effectiveness of the proposed algorithm. Results show that our approach allows the swarm to navigate without congestions in a smooth and coherent fashion, being suitable for large groups of robots.

Real-time search algorithms solve the problem of path planning, regardless the size and complexity of the maps, and the massive presence of entities in the same environment. In such methods, the learning step aims to avoid local minima and improve the results for future searches, ensuring the convergence to the optimal path when the same planning task is solved repeatedly. However, performing search in a limited area due to real-time constraints makes the run to convergence a lengthy process. In this work, we present a parallelization strategy that aims to reduce the time to convergence, maintaining the real-time properties of the search. The parallelization technique consists on using auxiliary searches without the real-time restrictions present in the main search. In addition, the same learning is shared by all searches. The empirical evaluation shows that even with the additional cost required to coordinate the auxiliary searches, the reduction in time to convergence is significant, showing gains from searches occurring in environments with fewer local minima to larger searches on complex maps, where performance improvement is even better.

This paper explores the use of Semiotic Engineering theory in the design of different interfaces for the control of a mobile robot. Based on a set of sign classes for human-robot interaction, we have elaborated a common interaction model and designed different interfaces for controlling an e-puck robot, a small sized mobile robot used for education and research. We implemented three interfaces using different technologies (desktop, Tablet PC, and handheld) and tested them with a group of users, in order to evaluate if the sign classes and their representations were adequate and also the differences perceived in the use of diverse interaction technologies.

A very common problem in the navigation of robotic swarms is when groups of robots move into opposite directions, causing congestion situations that may compromise performance. In this paper, we propose a distributed coordination algorithm to alleviate this type of congestion. By working collaboratively, and warning their teammates about a congestion risk, robots are able to coordinate themselves to avoid these situations. We executed simulations and real experiments to study the performance and effectiveness of the proposed algorithm. Results show that the algorithm allows the swarm to navigate in a smoother and more efficient fashion, and is suitable for large groups of robots.

One of the main problems in the navigation of robotic swarms is when several robots try to reach the same target at the same time, causing congestion situations that may compromise performance. In this paper, we propose a distributed coordination algorithm to alleviate this type of congestion. Using local sensing and communication, and controlling their actions using a probabilistic finite state machine, robots are able to coordinate themselves to avoid these situations. Simulations and real experiments were executed to study the performance and effectiveness of the proposed algorithm. Results show that the algorithm allows the swarm to have a more efficient and smoother navigation and is suitable for large groups of robots.

In this paper, we address navigation and coordination methods that allow swarms of robots to converge and spread along complex 2D shapes in environments containing unknown obstacles. Shapes are modeled using implicit functions and a gradient descent approach is used for controlling the swarm. To overcome local minima, that may appear in these scenarios, we use a coordination mechanism that reallocates some robots as “rescuers” and sends them to help other robots that may be trapped. Simulations and real experiments demonstrate the feasibility of the proposed approach.

We address the synthesis of controllers for large groups of robots and sensors, tackling the specific problem of controlling a swarm of robots to generate patterns specified by implicit functions of the form s(x, y) = 0. We derive decentralized controllers that allow the robots to converge to a given curve S and spread along this curve. We consider implicit functions that are weighted sums of radial basis functions created by interpolating from a set of constraint points, which give us a high degree of control over the desired 2D curves. We describe the generation of simple plans for swarms of robots using these functions and illustrate our approach through simulations and real experiments.

This paper addresses the problem of coordinating aerial and ground vehicles in tasks that involve exploration, identification of targets and maintaining a connected communication network. We focus on the problem of localizing vehicles in urban environments where GPS signals are often unreliable or unavailable. We first describe our multi-robot testbed and the control software used to coordinate ground and aerial vehicles. We present the results of experiments in air-ground localization analyzing three complementary approaches to determining the positions of vehicles on the ground. We show that the coordination of aerial vehicles with ground vehicles is necessary to get accurate estimates of the state of the system.

This paper proposes a methodology that uses hybrid systems to model multiple robots in the execution of cooperative tasks. Basically, each robot is represented by a hybrid automation and the cooperative task execution is modeled by the composition of several automata. We describe in details our approach to perform the composition of these automata and demonstrate the effectiveness of the proposed methodology modeling a cooperative manipulation task.

This paper presents ROCI, a framework for developing applications for multi-robot teams. In ROCI, each robot is considered a node, which contains several modules and may export different types of services and capabilities to other nodes. Each node runs a kernel that mediates the interactions of the robots in a team. This kernel keeps an updated database of all nodes and the functionalities that they export. Multi-robot applications can be built dynamically by connecting modules that may be running on different nodes over the network. As an example, we present an obstacle avoidance task implemented using our framework and also discuss the use of ROCI in a multi-robot scenario.

Proposes a methodology for coordinating multi-robot teams in the execution of cooperative tasks. It is based on a dynamic role-assignment mechanism in which the robots assume and exchange roles during cooperation. We model the role assignment under a hybrid systems framework, using a hybrid automaton to represent roles, transitions and controllers. Using a multi-robot simulator, the methodology is demonstrated in a cooperative transportation task, in which a group of robots must find and cooperatively transport several objects scattered in the environment.

Addresses the problem of coordinating multiple mobile robots in a tightly coupled task by means of implicit communication. This approach allows the development of controllers that do not depend on any explicit data flow between the robots, thus relying only on local sensor information. A box-carrying task is used to validate the proposed methodology both in simulation and in real-world experiments. Results show that implicit communication can be used together or replacing explicit communication for the cooperative box carrying task, making the system more robust to faulty communication environments.

Proposes an architecture for tightly coupled multi-robot coordination that is well suited to cooperative manipulation tasks. At all times, a robot is identified as a leader, while the others are designated as followers. The assignment of roles and the coordination between the robots is guaranteed by communication protocols and control algorithms. The key feature is the flexibility that allows changes in leadership and assignment of roles during the execution of a task. We describe the experimental implementation and demonstration in a cooperative transportation task, in which two and three heterogeneous robots cooperate to carry a large object in an environment containing obstacles.