In the modern manufacturing industry, workers are still required to manually perform complex, repetitive, and physically demanding tasks. Collaborative robotics has emerged to assist human workers, reducing physical strain and monotony, while increasing productivity and safety. Nonetheless, cobots still struggle to match the dexterity of human hands and they lack the ability to understand natural language and interpret human needs, leading to worker frustration and stress. Psychological stress is a critical issue in industrial workplaces, as it affects both workers’ well-being and productivity. This work addresses the issue of operators’ well-being in the context of human-robot collaboration within industrial settings. We propose a multimodal approach to detect psychological stress during an industrial assembly task. Data are collected from 12 participants while performing both autonomous and robot-assisted assembly tasks. Our approach combines physiological signals (ECG, EMG, EDA), facial action units (AUs), and voice features to classify stress levels. The extracted features are combined using a late fusion approach involving the use of a self-attention layer. The results demonstrate the effectiveness of our model in predicting stress levels with a weighted F1-score of 0.81. This research paves the way for the development of more empathetic and human-aware robotic partners, capable of adapting their behavior to improve collaboration and operator well-being.

We introduce PointIt, a toolkit for the Robot Operating System (ROS2) to build human-robot interfaces based on pointing gestures sensed by a wrist-worn Inertial Measurement Unit, such as a smartwatch. We release the software as open-source with MIT license; docker images and exhaustive instructions simplify its usage in simulated and real-world deployments.

Assessing the traversability of rugged terrain is a difficult challenge for legged robots, especially when they implement multiple, distinct gaits. We tackle this problem on the k-rock2 amphibious, sprawling gait robot by training a gait-dependent traversability estimator. We verify that the estimator, trained solely on procedurally-generated simulated data, approaches the outcomes of real-world experiments conducted in an indoor motion capture arena using two distinct terrestrial gaits to cross various indoor obstacles. In simulation experiments on a large-scale outdoor heightmap representing real-world data, we quantify the performance gain using the estimator outputs for gait selection. Further, we apply the method to heightmaps of outdoor data to illustrate how the approach could readily be applied to field scenarios.

Many emerging applications of nano-sized unmanned aerial vehicles (UAVs), with a few cm2form-factor, revolve around safely interacting with humans in complex scenarios, for example, monitoring their activities or looking after people needing care. Such sophisticated autonomous functionality must be achieved while dealing with severe constraints in payload, battery, and power budget (~100mW). In this work, we attack a complex task going from perception to control: to estimate and maintain the nano-UAV’s relative 3-D pose with respect to a person while they freely move in the environment—a task that, to the best of our knowledge, has never previously been targeted with fully onboard computation on a nano-sized UAV. Our approach is centered around a novel vision-based deep neural network (DNN), called Frontnet, designed for deployment on top of a parallel ultra-low power (PULP) processor aboard a nano-UAV. We present a vertically integrated approach starting from the DNN model design, training, and dataset augmentation down to 8-bit quantization and deployment in-field. PULP-Frontnet can operate in real-time (up to135 frame/s), consuming less than87 mWfor processing at peak throughput and down to0.43 mJ/framein the most energy-efficient operating point. Field experiments demonstrate a closed-loop top-notch autonomous navigation capability, with a tiny 27-g Crazyflie 2.1 nano-UAV. Compared against an ideal sensing setup, onboard pose inference yields excellent drone behavior in terms of median absolute errors, such as positional (onboard:41cm, ideal:26 cm) and angular (onboard:3.7°, ideal:4.1°). We publicly release videos and the source code of our work.

Deep neural networks (DNNs) are becoming the first-class solution for autonomous unmanned aerial vehicles (UAVs) applications, especially for tiny, resource-constrained, nano-UAVs, with a few tens of grams in weight and subten centimeters in diameter. DNN visual pipelines have been proven capable of delivering high intelligence aboard nanoUAVs, efficiently exploiting novel multi-core microcontroller units. However, one severe limitation of this class of solutions is the generalization challenge, i.e., the visual cues learned on the specific training domain hardly predict with the same accuracy on different ones. Ultimately, it results in very limited applicability of State-of-the-Art (SoA) autonomous navigation DNNs outside controlled environments. In this work, we tackle this problem in the context of the human pose estimation task with a SoA vision-based DNN [1]. We propose a novel methodology that leverages synthetic domain randomization by applying a simple but effective image background replacement technique to augment our training dataset. Our results demonstrate how the augmentation forces the learning process to focus on what matters most: the pose of the human subject. Our approach reduces the DNN’s mean square error — vs. a non-augmented baseline — by up to 40%, on a never-seen-before testing environment. Since our methodology tackles the DNN’s training stage, the improved generalization capabilities come at zero-cost for the computational/memory burdens aboard the nano-UAV.

We propose a practical approach for detecting the event that a human wearing an IMU-equipped bracelet points at a moving robot; the approach uses a learned classifier to verify if the robot motion (as measured by its odometry) matches the wrist motion, and does not require that the relative pose of the operator and robot is known in advance. To train the model and validate the system, we collect datasets containing hundreds of real-world pointing events. Extensive experiments quantify the performance of the classifiers and relevant metrics of the resulting detectors; the approach is implemented in a real-world demonstrator that allows users to land quadrotors by pointing at them.

We present an approach for controlling the position of a quadrotor in 3D space using pointing gestures; the task is difficult because it is in general ambiguous to infer where, along the pointing ray, the robot should go. We propose and validate a pragmatic solution based on a push button acting as a simple additional input device which switches between different virtual workspace surfaces. Results of a study involving ten subjects show that the approach performs well on a challenging 3D piloting task, where it compares favorably with joystick control.

We demonstrate a self-supervised approach which learns to detect long-range obstacles from video: it automatically obtains training labels by associating the camera frames acquired at a given pose to short-range sensor readings acquired at a different pose.

We propose a system to control robots in the users proximity with pointing gestures-a natural device that people use all the time to communicate with each other. Our system has two requirements: first, the robot must be able to reconstruct its own motion, e.g. by means of visual odometry; second, the user must wear a wristband or smartwatch with an inertial measurement unit. Crucially, the robot does not need to perceive the user in any way. The resulting system is widely applicable, robust, and intuitive to use.

We demonstrate a system to control robots in the users proximity with pointing gestures-a natural device that people use all the time to communicate with each other. Our setup consists of a miniature quadrotor Crazyflie 2.0, a wearable inertial measurement unit MetaWearR+ mounted on the user's wrist, and a laptop as the ground control station. The video of this demo is available at https://youtu.be/yafy-HZMk_U [1].

We consider a quadrotor equipped with a forward-facing camera, and an user freely moving in its proximity; we control the quadrotor in order to stay in front of the user, using only camera frames. To do so, we train a deep neural network to predict the drone controls given the camera image. Training data is acquired by running a simple hand-designed controller which relies on optical motion tracking data.

We present a system for interaction between co-located humans and mobile robots, which uses pointing gestures sensed by a wrist-mounted IMU. The operator begins by pointing, for a short time, at a moving robot. The system thus simultaneously determines: that the operator wants to interact; the robot they want to interact with; and the relative pose among the two. Then, the system can reconstruct pointed locations in the robot's own reference frame, and provide real-time feedback about them so that the user can adapt to misalignments. We discuss the challenges to be solved to implement such a system and propose practical solutions, including variants for fast flying robots and slow ground robots. We report different experiments with real robots and untrained users, validating the individual components and the system as a whole.

We consider the problem of planning paths on graphs with some edges whose traversability is uncertain; for each uncertain edge, we are given a probability of being traversable (e.g., by a learned classifier). We categorize different interpretations of the problem that are meaningful for mobile robots navigating partially-known environments, each of which yields a different formalization; we then focus on the case in which the true traversability of an edge is revealed only when the agent visits one of its endpoints (Canadian Traveller Problem). In this context, we design a large simulation campaign on synthetic and real-world maps to study the impact of two different factors: the planning strategy, and the amount of uncertainty (which could depend on the quality of the classifier producing traversability estimates).

We consider the task of controlling a quadrotor to hover in front of a freely moving user, using input data from an onboard camera. On this specific task we compare two widespread learning paradigms: a mediated approach, which learns a high-level state from the input and then uses it for deriving control signals; and an end-to-end approach, which skips high-level state estimation altogether. We show that despite their fundamental difference, both approaches yield equivalent performance on this task. We finally qualitatively analyze the behavior of a quadrotor implementing such approaches.

We propose a learning-based system for detecting when a user performs a pointing gesture, using data acquired from IMU sensors, by means of a 1D convolutional neural network. We quantitatively evaluate the resulting detection accuracy, and discuss an application to a human-robot interaction task where pointing gestures are used to guide a quadrotor landing.

Thymio is a small, inexpensive, mass-produced mobile robot with widespread use in primary and secondary education. In order to make it more versatile and effectively use it in later educational stages, including university levels, we have expanded Thymio's capabilities by adding off-the-shelf hardware and open software components. The resulting robot, that we call Mighty Thymio, provides additional sensing functionalities, increased computing power, networking, and full ROS integration. We present the architecture of Mighty Thymio and show its application in advanced educational activities.

We detect and localize obstacles in front of a mobile robot by means of a deep neural network that maps images acquired from a forward-looking camera to the outputs of five proximity sensors. The robot autonomously acquires training data in multiple environments; once trained, the network can detect obstacles and their position also in unseen scenarios, and can be used on different robots, not equipped with proximity sensors. We demonstrate both the training and deployment phases on a small modified Thymio robot.

We propose a model of artificial emotions for adaptation and implicit coordination in multi-robot systems. Artificial emotions play two roles, which resemble their function in animals and humans: modulators of individual behavior, and means of communication for social coordination. Emotions are modeled as compressed representations of the internal state, and are subject to a dynamics depending on internal and external conditions. Being a compressed representation, they can be efficiently exposed to nearby robots, allowing to achieve local group-level communication. The model is instantiated for a navigation task, with the aim of showing how coordination can effectively emerge by adding artificial emotions on top of an existing navigation framework. We show the positive effects of emotion-mediated group behaviors in a few challenging scenarios that would otherwise require ad hoc strategies: preventing deadlocks in crowded conditions; enabling efficient navigation of agents with time-critical tasks; assisting robots with faulty sensors. Two performance measures, throughput and number of collisions, are used to quantify the contribution of emotions for modulation and coordination.

We propose a novel approach to establish the relative pose of a mobile robot with respect to an operator that wants to interact with it; we focus on scenarios in which the robot is in the same environment as the operator, and is visible to them. The approach is based on comparing the trajectory of the robot, which is known in the robot's odometry frame, to the motion of the arm of the operator, who, for a short time, keeps pointing at the robot they want to interact with. In multi-robot scenarios, the same approach can be used to simultaneously identify which robot the operator wants to interact with. The main advantage over alternatives is that our system only relies on the robot's odometry, on a wearable inertial measurement unit (IMU), and, crucially, on the operator's own perception. We experimentally show the feasibility of our approach using real-world robots.

In the context of coordination of mobile multi-robot/agent networked teams, we present an integrated model that simultaneously addresses two problems arising in multi-robot missions: (i) task allocation and task scheduling; and (ii) communication provisioning in the multi-hop mobile ad hoc network built by the team. The integrated model is based on a mixed integer linear programming (MILP) formulation, which is solved in a centralized mode. For the communication part, the model solution outputs data routing policies and data transmission schedules that are aimed to maximize data delivery throughput to/from control centers. The trade-off between task and network performance optimization is strategically controlled. A refinement procedure is defined that allows to further improve communications by also minimizing network delays. We report a computational analysis of the integrated MILP model and an evaluation of the impact of a number of parameters on the trade-off between computational load and quality of the output. Results show that the model is computationally affordable for reasonably sized scenarios, and can effectively balance different performance trade-offs.

Cooperative Multi-Robot Observation of Multiple Moving Targets (CMOMMT) denotes a class of problems in which a set of autonomous mobile robots equipped with limited-range sensors are used to keep under observation a (possibly larger) set of mobile targets. Robots cooperatively plan their motion in order to maximize the time during which each target lies within the sensing range of at least one robot.

Human-swarm interaction (HSI) is a developing field of research in which the problem of gesture-based control has been attracting an increasing attention, being at the same time a natural form of interaction and an effective way to point and select individual or groups of robots in the swarm. Gesture-based interaction usually requires vision-based recognition and classification of the gesture from the swarm. At this aim, existing methods for cooperative sensing and recognition make use of distributed consensus algorithms, which include for instance averaging and frequency counting. In this work we present a distributed consensus protocol that allows robot swarms to learn efficiently gestures from online interactions with a human teacher. The protocol also facilitates the integration of different consensus algorithms. Experiments have been performed in emulation using on real data acquired by a swarm of robots. The results indicate that effectively exploiting the collective decision-making of the swarm is a viable way to rapidly achieve good learning performance.

No abstract available.

As a first step towards human and multiple-UAV interaction, we present a novel method for humans to interact with flying UAVs using locally on-board video cameras. Using machine vision techniques, our approach enables human operators to command and control Parrot drones by giving them directions to move, using simple hand gestures. When a direction to move is given, the robot controller estimates the angle and distance to move with the help of a face score system and the estimated hand direction. This approach offers mobile robots the ability localize with human operators and provides UAVs/UGVs with a better perception of the environment around the human.

No abstract available.

Human localization is fundamental in human centered computing and human-robot interaction (HRI), as human operators should be localized by robots before being actively serviced. This paper proposes a simple and efficient approach for estimating the distance and orientation of an human, from a single robot-acquired image. We adopt a simple combination of multiple Haar feature-based classifiers to compute face scores, that represent the probability that the detected face is acquired from each of a predefined set of poses. Using the Locally Weighted Projectron Regression (LWPR), an online incremental regression-based learning scheme, we can reliably learn and predict the pose of a human face in real-time at a low computational cost. The accuracy, robustness, and scalability of the obtained solutions have been verified through emulation experiments performed on a large data set of real data acquired by a networked swarm of robots.

The performance of a team of robots executing a coordination task is, to a large extent, determined by the reliability of the communications between the robots. In wireless networks, one way to improve this reliability is to choose the best among the available wireless links. For this purpose, an accurate link quality model is required. We show how a group of robots can exploit their mobility to effectively and rapidly learn such a model directly from an unknown environment. The LQE (Link Quality Estimation) protocol, which is used by the robots to cooperatively collect labeled link quality samples, and learn out of them, is presented in the paper. The accuracy and the robustness of the LQE approach are validated through a set of real-world experiments, performed with mobile robots operating in different network environments. Moreover, in simulation, we study a multi-robot coordination problem, and show the benefits of using the link quality learning approach, at the expenses of devoting little time for learning the model before executing the task.

We present a novel approach for the online learning of hand gestures in swarm robotic (multi-robot) systems. We address the problem of online feature learning by proposing Convolutional Max-Pooling (CMP), a simple feed-forward two-layer network derived from the deep hierarchical Max-Pooling Convolutional Neural Network (MPCNN). To learn and classify gestures in an online and incremental fashion, we employ a 2nd order online learning method, namely the Soft-Confidence Weighted (SCW) learning scheme. In order for all robots to collectively take part in the learning and recognition task and obtain a swarm-level classification, we build a distributed consensus by fusing the individual decision opinions of robots together with the individual weights generated from multiple classifiers. Accuracy, robustness, and scalability of obtained solutions have been verified through emulation experiments performed on a large data set of real data acquired by a networked swarm of robots.

We consider the problem of the joint mission planning in teams of heterogeneous physical agents. The type of missions that we consider are composed of spatially distributed tasks that need to be selected and assigned to the agents for dealing with them. A plan consists of a set of directives specifying who does what, where, when, and for how long. The aim is to optimize system-level performance by explicitly taking into account and exploiting the different sensory-motor characteristics of the agents. We tackle this general problem by proposing a mixed integer linear formulation of it which includes several aspects/constraints that closely model features and requests of realistic scenarios. We present a solution approach based on the combination of a metaheuristic and mathematical programming method that allows to compute high-quality plans within short time, and with formal guarantees on their optimality. The application of the framework is validated in the context of search and rescue missions.

Once a multi-robot system is implemented on real hardware and tested in the real world, analyzing its evolution and debugging unexpected behaviors is often a very difficult task. We present a tool for aiding this activity, by visualizing an Augmented Reality overlay on a live video feed acquired by a fixed camera overlooking the robot environment. Such overlay displays live information exposed by each robot, which may be textual (state messages), symbolic (graphs, charts), or, most importantly, spatially-situated; spatially-situated information is related to the environment surrounding the robot itself, such as for example the perceived position of neighboring robots, the perceived extent of obstacles, the path the robot plans to follow. We show that, by directly representing such information on the environment it refers to, our proposal removes a layer of indirection and significantly eases the process of understanding complex multi-robot systems. We describe how the system is implemented, discuss application examples in different scenarios, and provide supplementary material including demonstration videos and a functional implementation.

Small-footprint mobile ground robots, such as the popular Turtlebot and Kobuki platforms, are by necessity equipped with sensors which lie close to the ground. Reliably detecting and tracking people from this viewpoint is a challenging problem, whose solution is a key requirement for many applications involving sharing of common spaces and close human-robot interaction. We present a robust solution for cluttered indoor environments, using an inexpensive RGB-D sensor such as the Microsoft Kinect or Asus Xtion. Even in challenging scenarios with multiple people in view at once and occluding each other, our system solves the person detection problem significantly better than alternative approaches, reaching a precision, recall and F1-score of 0.85, 0.81 and 0.83, respectively. Evaluation datasets, a real-time ROS-enabled implementation and demonstration videos are provided as supplementary material.

This paper presents a machine vision based approach for human operators to select individual and groups of autonomous robots from a swarm of UAVs. The angular distance between the robots and the human is estimated using measures of the detected human face, which aids to determine human and multi-UAV localization and positioning. In turn, this is exploited to effectively and naturally make the human select the spatially situated robots. Spatial gestures for selecting robots are presented by the human operator using tangible input devices (i.e., colored gloves). To select individuals and groups of robot we formulate a vocabulary of two-handed spatial pointing gestures. With the use of a Support Vector Machine (SVM) trained in a cascaded multi-binary-class configuration, the spatial gestures are effectively learned and recognized by a swarm of UAVs.

Deep Neural Networks now excel at image classification, detection and segmentation. When used to scan images by means of a sliding window, however, their high computational complexity can bring even the most powerful hardware to its knees. We show how dynamic programming can speedup the process by orders of magnitude, even when max-pooling layers are present.

The vision-based mechanisms that pedestrians in social groups use to navigate in dynamic environments, avoiding obstacles and each others, have been subject to a large amount of research in social anthropology and biological sciences. We build on recent results in these fields to develop a novel fully-distributed algorithm for robot local navigation, which implements the same heuristics for mutual avoidance adopted by humans. The resulting trajectories are human-friendly, because they can intuitively be predicted and interpreted by humans, making the algorithm suitable for the use on robots sharing navigation spaces with humans. The algorithm is computationally light and simple to implement. We study its efficiency and safety in presence of sensing uncertainty, and demonstrate its implementation on real robots. Through extensive quantitative simulations we explore various parameters of the system and demonstrate its good properties in scenarios of different complexity. When the algorithm is implemented on robot swarms, we could observe emergent collective behaviors similar to those observed in human crowds.

We present the Swiss2Grid project, a pilot and demonstration aimed at evaluating the impact of different distributed demand management policies in Smart Grids. The increasing diffusion of decentralised energy generation, especially photovoltaics, can lead to severe imbalances on the electric grid, which could require huge investments in grid infrastructures. The approach proposed by the Swiss2Grid project is to adopt a decentralised approach to load management at the local level. Single households use a local algorithm that, based only on local voltage and frequency measures, shifts the pre-emptible loads in time in order to minimise the costs for the consumer and to maximise the grid stability. In this paper we present the project set-up in Mendrisio, a city in Southern Switzerland, we describe the algorithm principles, and finally we present some preliminary results showing the impact of the Swiss2Grid algorithm on the Low Voltage grid.

Most local robot navigation algorithms are based on the concept of velocity obstacle, a mechanistic approach to the navigation problem in which a solution is engineered from scratch. Over the years, a number of different velocity obstacle variants have been developed to effectively handle multi-robot systems. In parallel, an alternative, human-inspired approach for robot navigation has been recently proposed, which derives from the observation and modeling of crowds of pedestrians. We discuss similarities and differences among two broadly used obstacle-velocity variants, namely Hybrid Reciprocal Velocity Obstacle and Optimal Reciprocal Collision Avoidance, and the human-inspired approach. How do these differences (which are often subtle) impact performance, and why? We answer these questions through extensive simulation experiments, wherein we evaluate the the algorithms for safety, trajectory efficiency, and emergence of collective behaviors, in different challenging multi-robot scenarios using both ideal and realistic models for robots and sensing.

In transportation problems like the vehicle routing problem, the decision makers are increasingly adopting the idea that the problem data can be subject to uncertainty. The uncertainty can be encountered because of events that are not exactly predictable, like weather conditions, traffic jams, etc. In this paper, we study vehicle routing problem with uncertain travel costs. Then, to solve the problem, we propose a robust multiple ant colony system: a metaheuristic in which multiple ant colonies work in parallel to generate a collection of solutions with different levels of protection against the uncertainty. The uncertainty is handled by incorporating linear formulations from the field of robust optimization into the metaheuristic approach.

In this study, we consider a capacitated vehicle routing problem where the objective function is to minimize the total travel cost.We also consider that the travel costs between the locations are subject to uncertainty, therefore they are expressed as intervals, rather than fixed numbers. The motivation of this study is to solve this problem by using a metaheuristic approach. We base our approach on a variant of ant colony optimization metaheuristic, called ant colony system, which was originally implemented for solving the deterministic version of the problem (i.e. the classical version of the problem without the uncertainty), previously reported in the literature. We modify the algorithm to incorporate a robust optimization methodology, so that the uncertainty on traveling costs can be handled.

No abstract available.

We introduce Convolutional Neural Support Vector Machines (CNSVMs), a combination of two heterogeneous supervised classification techniques, Convolutional Neural Networks (CNNs) and Support Vector Machines (SVMs). CNSVMs are trained using a Stochastic Gradient Descent approach, that provides the computational capability of online incremental learning and is robust for typical learning scenarios in which training samples arrive in mini-batches. This is the case for visual learning and recognition in multi-robot systems, where each robot acquires a different image of the same sample. The experimental results indicate that the CNSVM can be successfully applied to visual learning and recognition of hand gestures as well as to measure learning progress.

We present an approach for distributed real-time recognition tasks using a swarm of mobile robots. We focus on the visual recognition of hand gestures, but the solutions that we provide have general applicability and address a number of challenges common to many distributed sensing and classification problems. In our approach, robots acquire and process hand images from multiple points of view, most of which do not allow for a satisfactory classification. Each robot is equipped with a statistical classifier, which is used to generate an opinion for the sensed gesture. Using a low-bandwidth wireless channel, the robots locally exchange their opinions. They also exploit mobility to adapt their positions to maximize the mutual information collectively gathered by the swarm. A distributed consensus protocol is implemented, to allow to rapidly settle on a decision once enough evidence is available. The system is implemented and demonstrated on real robots. In addition, extensive quantitative results of emulation experiments, based on a real image dataset, are reported. We consider different scenarios and study the scalability and the robustness of the swarm performance for distributed recognition.

Networked robotics is an area that integrates multi-robot and network technology. The characteristics and the reliability of the communication environment play a fundamental role shaping and affecting behavior and performance of a mobile multi-robot system. In this context, two basic questions arise: how much the overall performance is affected and how can we investigate this influence? Addressing these two questions, in this paper we present the architecture of an integrated simulation environment that allows for realistic simulation of networked robotic systems. The proposed framework integrates two simulators: a network simulator and a multi-robot simulator. We present two implementations based on the ARGoS simulator for the robotic side, and with ns-2 and ns-3 employed as network simulators. We evaluate the proposed tools, both in isolation and integration, and show that they are able to efficiently simulate systems consisting of hundreds of robots. Moreover, we use the proposed framework to demonstrate the effects of communication on the performance of a mobile multi-robot system performing distributed coordination and task assignment. We compare realistic network simulation with simplified communication models and we study the resulting behavior and performance of the robotic system.

We address a central problem of neuroanatomy, namely, the automatic segmentation of neuronal structures depicted in stacks of electron microscopy (EM) images. This is necessary to efficiently map 3D brain structure and connectivity. To segment {\em biological} neuron membranes, we use a special type of deep {\em artificial} neural network as a pixel classifier. The label of each pixel (membrane or non-membrane) is predicted from raw pixel values in a square window centered on it. The input layer maps each window pixel to a neuron. It is followed by a succession of convolutional and max-pooling layers which preserve 2D information and extract features with increasing levels of abstraction. The output layer produces a calibrated probability for each class. The classifier is trained by plain gradient descent on a $512 \times 512 \times 30$ stack with known ground truth, and tested on a stack of the same size (ground truth unknown to the authors) by the organizers of the ISBI 2012 EM Segmentation Challenge. Even without problem-specific post-processing, our approach outperforms competing techniques by a large margin in all three considered metrics, i.e. \emph{rand error}, \emph{warping error} and \emph{pixel error}. For pixel error, our approach is the only one outperforming a second human observer.

In this paper we consider a human-swarm interaction scenario based on hand gestures. We study how the swarm can incrementally learn hand gestures through the interaction with a human instructor providing training gestures and correction feedback. The main contribution of the paper is a novel incremental machine learning approach that makes the robot swarm learn and recognize the gestures in a distributed and decentralized fashion using binary (i.e., yes/no) feedback. It exploits cooperative information exchange and swarm's intrinsic parallelism and redundancy. We perform extensive tests using real gesture images, showing that good classification accuracies are obtained even with rather few training samples and relatively small swarms. We also show the good scalability of the approach and its relatively low requirements in terms of communication overhead.

In 2010, after many years of stagnation, the MNIST handwriting recognition benchmark record dropped from 0.40% error rate to 0.35%. Here we report 0.27% for a committee of seven deep CNNs trained on graphics cards, narrowing the gap to human performance. We also apply the same architecture to NIST SD 19, a more challenging dataset including lower and upper case letters. A committee of seven CNNs obtains the best results published so far for both NIST digits and NIST letters. The robustness of our method is verified by analyzing 78125 different 7-net committees.

We present a new method to train the members of a committee of one-hidden-layer neural nets. Instead of training various nets on subsets of the training data we preprocess the training data for each individual model such that the corresponding errors are decor related. On the MNIST digit recognition benchmark set we obtain a recognition error rate of 0.39%, using a committee of 25 one-hidden-layer neural nets, which is on par with state-of-the-art recognition rates of more complicated systems.

We present a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants. Our feature extractors are neither carefully designed nor pre-wired, but rather learned in a supervised way. Our deep hierarchical architectures achieve the best published results on benchmarks for object classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple back-propagation perform better than more shallow ones. Learning is surprisingly rapid. NORB is completely trained within five epochs. Test error rates on MNIST drop to 2.42%, 0.97 % and 0.48 % after 1, 3 and 17 epochs, respectively.

We present a communication based navigation algorithm for robotic swarms. It lets robots guide each other's navigation by exchanging messages containing navigation information through the wireless network formed among the swarm. We study the use of this algorithm in two different scenarios. In the first scenario, the swarm guides a single robot to a target, while in the second, all robots of the swarm navigate back and forth between two targets. In both cases, the algorithm provides efficient navigation, while being robust to failures of robots in the swarm. Moreover, we show that in the latter case, the system lets the swarm self-organize into a robust dynamic structure. This self-organization further improves navigation efficiency, and is able to find shortest paths in cluttered environments. We test our system both in simulation and on real robots.

We present ARGoS, a novel open source multi-robot simulator. The main design focus of ARGoS is the real-time simulation of large heterogeneous swarms of robots. Existing robot simulators obtain scalability by imposing limitations on their extensibility and on the accuracy of the robot models. By contrast, in ARGoS we pursue a deeply modular approach that allows the user both to easily add custom features and to allocate computational resources where needed by the experiment. A unique feature of ARGoS is the possibility to use multiple physics engines of different types and to assign them to different parts of the environment. Robots can migrate from one engine to another transparently. This feature enables entirely novel classes of optimizations to improve scalability and paves the way for a new approach to parallelism in robotics simulation. Results show that ARGoS can simulate about 10,000 simple wheeled robots 40% faster than real-time.

As microscopes have a very shallow depth of field, Z-stacks (i.e. sets of images shot at different focal planes) are often acquired to fully capture a thick sample. Such stacks are viewed by users by navigating them through the mouse wheel. We propose a new technique of visualizing 3D point, line or area markers in such focus stacks, by displaying them with a depth-dependent defocus, simulating the microscope's optics; this leverages on the microscopists' ability to continuously twiddle focus, while implicitly performing a shape-from-focus reconstruction of the 3D structure of the sample. User studies confirm that the approach is effective, and can complement more traditional techniques such as color-based cues. We provide two implementations, one of which computes defocus in real time on the GPU, and examples of their application.

We study how a swarm robotic system consisting of two different types of robots can solve a foraging task. The first type of robots are small wheeled robots, called foot-bots, and the second type are flying robots that can attach to the ceiling, called eye-bots. While the foot-bots perform the actual foraging, i.e. they move back and forth between a source and a target location, the eye-bots are deployed in stationary positions against the ceiling, with the goal of guiding the foot-bots. The key component of our approach is a process of mutual adaptation, in which foot-bots execute instructions given by eye-bots, and eye-bots observe the behavior of foot-bots to adapt the instructions they give. Through a simulation study, we show that this process allows the system to find a path for foraging in a cluttered environment. Moreover, it is able to converge onto the shortest of two paths, and spread over different paths in case of congestion. Since our approach involves mutual adaptation between two sub-swarms of different robots, we refer to it as cooperative self-organization. This is to our knowledge the first work that investigates such a system in swarm robotics.

Good old online backpropagation for plain multilayer perceptrons yields a very low 0.35% error rate on the MNIST handwritten digits benchmark. All we need to achieve this best result so far are many hidden layers, many neurons per layer, numerous deformed training images to avoid overfitting, and graphics cards to greatly speed up learning.

This special issue contains seven papers describing recent research developments in the swarm intelligence (SI) field.

We propose a new approach to integrate proactive and reactive routing in mobile ad hoc networks. Our work deploys a lightweight proactive algorithm that runs in the background offering a basic routing service, and a reactive algorithm that can be called on demand offering a connection-oriented service. The reactive algorithm uses the routing information from the proactive algorithm in its working, so that there is a synergy between the two parts of the system. An important property of our system is that it allows the choice between proactive and reactive routing to be made for each session individually, by their source nodes. This gives network nodes a very fine-grained level of control over the routing process, and allows them to exploit the complementary properties of proactive and reactive routing, e.g. by matching the choice of the routing approach to the needs of individual sessions. In a range of simulation tests, we evaluate the validity of our approach.

We study through simulation the performance of two swarm intelligence MANET routing algorithms in a realistic urban environment. The two algorithms, ANSI and AntHocNet, implement the swarm intelligence paradigm for routing in different ways: while ANSI applies a reactive approach in which ants are only sent out when no route is available between the source and destination of a communication session, AntHocNet integrates reactive and proactive mechanisms whereby the algorithm sends out ants at regular intervals during the entire duration of running sessions in order to continuously adapt and improve existing routes. The two swarm intelligence routing algorithms are compared to AODV, a state-of-the-art reactive algorithm, and OLSR, a state-of-the-art proactive algorithm. Our objective is to investigate the usefulness of the different approaches adopted by the algorithms when confronted with the peculiarities of urban environments and the requirements of real-world applications. At this aim we define a detailed and realistic simulation setup. We model node mobility by limiting node movements to the streets and open spaces of town, use a ray-tracing approach to model the propagation of radio waves, and investigate different kinds of interactive data traffic patterns, ranging from SMS messaging to VoIP communications.

The sequential ordering problem is a version of the asymmetric traveling salesman problem where precedence constraints on vertices are imposed. A tour is feasible if these constraints are respected, and the objective is to find a feasible solution with minimum cost. The sequential ordering problem models a lot of real world applications, mainly in the fields of transportation and production planning. In this paper we propose an extension of a well known ant colony system for the problem, aiming at making the approach more efficient on large problems. The extension is based on a problem manipulation technique that heuristically reduces the search space. Computational results, where the extended ant colony system is compared to the original one, are presented

Recent developments in information technology have brought about important changes in distributed computing. New environments such as massively large-scale, wide-area computer networks and mobile ad hoc networks have emerged. Common characteristics of these environments include extreme dynamicity, unreliability, and large scale. Traditional approaches to designing distributed applications in these environments based on central control, small scale, or strong reliability assumptions are not suitable for exploiting their enormous potential. Based on the observation that living organisms can effectively organize large numbers of unreliable and dynamically-changing components (cells, molecules, individuals, etc.) into robust and adaptive structures, it has long been a research challenge to characterize the key ideas and mechanisms that make biological systems work and to apply them to distributed systems engineering. In this article we propose a conceptual framework that captures several basic biological processes in the form of a family of design patterns. Examples include plain diffusion, replication, chemotaxis, and stigmergy. We show through examples how to implement important functions for distributed computing based on these patterns. Using a common evaluation methodology, we show that our bio-inspired solutions have performance comparable to traditional, state-of-the-art solutions while they inherit desirable properties of biological systems including adaptivity and robustness.

We present AntNet-QoS, a novel approach that taking inspiration from the Ant Colony Optimization (ACO) framework, and in particular from the AntNet routing algorithm, allows dynamic scheduling, forwarding and link sharing among multiple packet classes in a differentiated service (DiffServ) network.

Mobile ad hoc networks are communication networks built up of a collection of mobile devices, which can communicate through wireless connections. Routing is the task of directing data packets from a source node to a given destination. This task is particularly hard in mobile ad hoc networks: due to the mobility of the network elements and the lack of central control, routing algorithms should be robust, adaptive, and work in a decentralized and self-organizing way. In this paper, we describe an algorithm, which draws inspiration from swarm intelligence to obtain these characteristics. More specifically, we borrow ideas from ant colonies and from the ant colony optimization framework. In an extensive set of simulation tests, we compare our routing algorithm with a state-of-the-art algorithm, and show that it gets better performance over a wide range of different scenarios and for a number of different evaluation measures. In particular, we show that it scales better with the number of nodes in the network.

We consider the problem of assigning transmission powers to the nodes of a wireless network in such a way that all the nodes are connected by bidirectional links and the total power consumption is minimized. Since no central authority (with a global vision of the network) exists in wireless networks, only distributed, swarm approaches can be used. We present a distributed protocol that embeds well-known centralized techniques for power minimization, here used in a local, distributed fashion. The result can be seen as a complex adaptive system (the global network), where global optimization emerges as a result of the behavior of local nodes, each one carrying out a myopic, local optimization. Computational results, proving the effectiveness of the new protocol, are finally presented.

Broadcasting in wireless networks, unlike wired networks, inherently reaches several nodes with a single transmission. For an omnidirectional wireless broadcast to a node, all nodes closer to the transmitting node are also reached. This property can be used to compute routing trees which minimize the sum of the transmitter powers. We present a mixed integer programming formulation and a simulated annealing algorithm for the problem. Extensive experimental results for the heuristic approach are presented. They show that the proposed algorithm is capable of improving the results of state-of-the-art algorithms for most of the problems considered. The solutions provided by the simulated annealing algorithm can be improved by applying a very fast post-optimization procedure. This leads to the best known mean results for the problems considered.

This paper describes AntHocNet, an algorithm for routing in mobile ad-hoc networks based on ideas from the ant colony optimisation framework. In AntHocNet a source node reactively sets up a path to a destination node at the start of each communication session. During the course of the session, the source node uses ant agents to proactively search for alternatives and improvements of the original path. This allows to adapt to changes in the network, and to construct a mesh of alternative paths between source and destination. The proactive behaviour is supported by a lightweight information bootstrapping process. Paths are represented in the form of distance-vector routing tables called pheromone tables. An entry of a pheromone table contains the estimated goodness of going over a certain neighbour to reach a certain destination. Data are routed stochastically over the different paths of the mesh according to these goodness estimates. In an extensive set of simulation tests, we compare AntHocNet to AODV, a reactive algorithm which is an important reference in this research area. We show that AntHocNet can outperform AODV for different evaluation criteria in a wide range of different scenarios. AntHocNet is also shown to scale well with respect to the number of nodes.

In IP-over-WDM networks, a logical IP network has to be routed on top of a physical optical fiber network. An important challenge hereby is to make the routing survivable. We call a routing survivable if the connectivity of the logical network is guaranteed in case of a failure in the physical network. In this paper we describe FastSurv, a local search algorithm which can provide survivable routing in the presence of physical link failures. The algorithm can easily be extended for the case of node failures and multiple simultaneous link failures. In a large series of test runs, we show that FastSurv is much more scalable with respect to the number of nodes in the network than current state-of-the-art algorithms, both in terms of solution quality and run time.

In IP-over-WDM networks, a logical IP network has to be routed on top of a physical optical fiber network. An important challenge is to make this routing survivable. We call a routing survivable if no single physical link failure can disconnect the logical topology. In this paper we present FastSurv, a local search algorithm for survivable routing. FastSurv works in an iterated way: after each iteration it learns more about the structure of the logical graph and in the next iteration it uses this information to improve its solution. We also extend the algorithm to take link capacity constraints into account. We show that our simple algorithm can produce better and faster results than current state-of-the-art algorithms.

SCOPUS: ed.j

We present a new local optimizer called SOP-3-exchange for the sequential ordering problem that extends a local search for the traveling salesman problem to handle multiple constraints directly without increasing computational complexity. An algorithm that combines the SOP-3-exchange with an Ant Colony Optimization algorithm is described, and we present experimental evidence that the resulting algorithm is more effective than existing methods for the problem. The best-known results for many of a standard test set of 22 problems are improved using the SOP-3-exchange with our Ant Colony Optimization algorithm or in combination with the MPO/AI algorithm (Chen and Smith 1996).

This article presents an overview of recent work on ant algorithms, that is, algorithms for discrete optimization that took inspiration from the observation of ant colonies' foraging behavior, and introduces the ant colony optimization (ACO) metaheuristic. In the first part of the article the basic biological findings on real ants are reviewed and their artificial counterparts as well as the ACO metaheuristic are defined. In the second part of the article a number of applications of ACO algorithms to combinatorial optimization and routing in communications networks are described. We conclude with a discussion of related work and of some of the most important aspects of the ACO metaheuristic.

This paper introduces the ant colony system (ACS), a distributed algorithm that is applied to the traveling salesman problem (TSP). In the ACS, a set of cooperating agents called ants cooperate to find good solutions to TSPs. Ants cooperate using an indirect form of communication mediated by a pheromone they deposit on the edges of the TSP graph while building solutions. We study the ACS by running experiments to understand its operation. The results show that the ACS outperforms other nature-inspired algorithms such as simulated annealing and evolutionary computation, and we conclude comparing ACS-3-opt, a version of the ACS augmented with a local search procedure, to some of the best performing algorithms for symmetric and asymmetric TSPs.

In this work we propose a Lyapunov based design of velocity observers and the controller for stable trajectory tracking by a robotic manipulator. It is shown how the proposed design is exponentially stable over a finite domain, and, in the high gain approximation, exponentially stable in the large domain. Moreover, the design proposed leads naturally to a reduced order observer structure, with considerable implementation advantages.

Robot motion planning in a cluttered environment requires knowledge about robot shape and size. These robot characteristics influence system performance eventhough most motion planning methods do not consider them. This paper presents an ongoing work that studies general motion planning techniques in combination with knowledge related to robot shape and size. The system acquires knowledge and learns strategies to avoid local collitions and to make global decisions. A neural network is presented that learns local behavior and a learning technique based on a reinforcement method is presented to overcome problems of local minimum.

This paper proposes a new method for grasping an object positioned in a cluttered workspace. The grasping of an object usually relies on the analysis of its contour in order to determine the contact points for the fingers of the gripper. The planning of the positioning of the real gripper in the workspace at the chosen points is often considered as a separate problem. The method proposed in this paper plans the placing of the gripper onto the contour of the objects thereby considering the size and shape of the fingers. A collision-free grasp for a gripper is found by simulating an attractive motion of the gripper towards the object until a configuration is found that satisfies the stability requirement. The technique relies on artificial fields defined over the workspace which are used to guide the motion of the gripper and to evaluate grasp configurations. Appropriate strategies are used to overcome local minima during the search. 1 Introduction Most traditional grasp planners rely o...

An iterative learning technique is applied to robot manipulators, using an inherently nonlinear analysis of the learning procedure. In particularly, a 'high-gain feedback' point of view is utilized to prove the possibility of setting up uniform upper bounds to the trajectory errors occurring at each trial. The subsequent analysis of convergence shows that apart from minor conditions, the existence of a finite (but not necessarily narrow) bound on the trajectory deviations can substantially suffice to guarantee the zeroing of the errors after a sufficient number of trials. This in turn leaves open the possibility of obtained the exact tracking of the desired motion, even in the presence of moderate values assigned to the feedback gains.>

The paper presents the basic ideas underlying a learning methodology for the control of movements of robotic manipulators. It is based on the use of repeated trials of tracking of a preassigned trajectory. Sufficient conditions for the convergence of the algorithm are presented.