In this paper, we focus on the robot grasping problem with parallel grippers using image data. For this task, we propose and implement an end-to-end approach. In order to detect the good grasping poses for a parallel gripper from RGB images, we have employed transfer learning for a Convolutional Neural Network (CNN) based object detection architecture. Our obtained results show that, the adapted network either outperforms or is on-par with the state-of-the art methods on a benchmark dataset. We also performed grasping experiments on a real robot platform to evaluate our method's real world performance.

While today's robots are able to perform sophisticated tasks, they can only act on objects they have been trained to recognize. This is a severe limitation: any robot will inevitably see new objects in unconstrained settings, and thus will always have visual knowledge gaps. However, standard visual modules are usually built on a limited set of classes and are based on the strong prior that an object must belong to one of those classes. Identifying whether an instance does not belong to the set of known categories (i.e. open set recognition), only partially tackles this problem, as a truly autonomous agent should be able not only to detect what it does not know, but also to extend dynamically its knowledge about the world. We contribute to this challenge with a deep learning architecture that can dynamically update its known classes in an end-to-end fashion. The proposed deep network, based on a deep extension of a non-parametric model, detects whether a perceived object belongs to the set of categories known by the system and learns it without the need to retrain the whole system from scratch. Annotated images about the new category can be provided by an `oracle' (i.e. human supervision), or by autonomous mining of the Web. Experiments on two different databases and on a robot platform demonstrate the promise of our approach.

Robotic technology has transformed manufacturing industry ever since the first industrial robot was put in use in the beginning of the 60s. The challenge of developing flexible solutions where production lines can be quickly re-planned, adapted and structured for new or slightly changed products is still an important open problem. Industrial robots today are still largely preprogrammed for their tasks, not able to detect errors in their own performance or to robustly interact with a complex environment and a human worker. The challenges are even more serious when it comes to various types of service robots. Full robot autonomy, including natural interaction, learning from and with human, safe and flexible performance for challenging tasks in unstructured environments will remain out of reach for the foreseeable future. In the envisioned future factory setups, home and office environments, humans and robots will share the same workspace and perform different object manipulation tasks in a collaborative manner. We discuss some of the major challenges of developing such systems and provide examples of the current state of the art.

Technological developments call for increasing perception and action capabilities of robots. Among other skills, vision systems that can adapt to any possible change in the working conditions are needed. Since these conditions are unpredictable, we need benchmarks which allow to assess the generalization and robustness capabilities of our visual recognition algorithms. In this work we focus on robotic kitting in unconstrained scenarios. As a first contribution, we present a new visual dataset for the kitting task. Differently from standard object recognition datasets, we provide images of the same objects acquired under various conditions where camera, illumination and background are changed. This novel dataset allows for testing the robustness of robot visual recognition algorithms to a series of different domain shifts both in isolation and unified. Our second contribution is a novel online adaptation algorithm for deep models, based on batch-normalization layers, which allows to continuously adapt a model to the current working conditions. Differently from standard domain adaptation algorithms, it does not require any image from the target domain at training time. We benchmark the performance of the algorithm on the proposed dataset, showing its capability to fill the gap between the performances of a standard architecture and its counterpart adapted offline to the given target domain.

Picking up objects requested by a human user is a common task in human-robot interaction. When multiple objects match the user's verbal description, the robot needs to clarify which object the user is referring to before executing the action. Previous research has focused on perceiving user's multimodal behaviour to complement verbal commands or minimising the number of follow up questions to reduce task time. In this paper, we propose a system for reference disambiguation based on visualisation and compare three methods to disambiguate natural language instructions. In a controlled experiment with a YuMi robot, we investigated realtime augmentations of the workspace in three conditions - head-mounted display, projector, and a monitor as the baseline - using objective measures such as time and accuracy, and subjective measures like engagement, immersion, and display interference. Significant differences were found in accuracy and engagement between the conditions, but no differences were found in task time. Despite the higher error rates in the head-mounted display condition, participants found that modality more engaging than the other two, but overall showed preference for the projector condition over the monitor and head-mounted display conditions.

In this paper, we present an object based approach for human-centric partitioning of the environment. Our approach for determining the human-centric regions is to detect the objects that are commonly associated with frequent human presence. In order to detect these objects, we employ state of the art perception techniques. The detected objects are stored with their spatio-temporal information in the robot's memory to be later used for generating the regions. The advantages of our method is that it is autonomous, requires only a small set of perceptual data and does not even require people to be present while generating the regions. The generated regions are validated using a 1-month dataset collected in an indoor office environment. The experimental results show that although a small set of perceptual data is used, the regions are generated at densely occupied locations.

This paper considers the merging of appearance-based spatial knowledge among robots having compatible visual sensing. Each robot is assumed to retain its knowledge in its individual long-term spatial memory where i) the place knowledge and their spatial relations are retained in an organized manner in place and map memories respectively; and ii) a ‘place’ refers to a spatial region as designated by a collection of associated appearances. In the proposed approach, each robot communicates with another robot, receives its memory and then merges the received knowledge with its own. The novelty of the merging process is that it is done in two stages: merging of place knowledge followed by the merging of map knowledge. As each robot's place memory is processed as a whole or in portions, the merging process scales easily with respect to the amount and overlap of the appearance data. Furthermore, the merging can be done in decentralized manner. Our experimental results with a team of three robots demonstrate that the resulting merged knowledge enables the robots to reason about learned places.

This paper introduces a novel approach to topo-logical place detection. The approach is based on previously proposed bubble space representation - where all sensory features and their relative S2- geometry are encoded in a manner that is implicitly dependent on robot pose. Its novelty is that ensuring sensory data reliability is integrated with place detection. This is achieved via checking for informativeness, coherence and plenitude using only the bubble space representation of the incoming sensory data. The stringency of these checks is controllable via a set of associated parameters. Experimental results with benchmark datasets indicate correct detection rates comparable to state-of-the-art approaches in place detection. Furthermore, the detected places can then be immediately used to generate the nodes in topological maps.

Multi-robot coordination is one of the hot topics of today's robotics research. In our case, our main concern is to build up a small sized (SSL) robot soccer team. Robot soccer requires effective and fast robot control and coordination in order to score goals while avoiding frequent collisions with opponent robots. In this paper, our multi-robot platform and current control techniques we use are discussed. Our robots have 4 wheel omnidirectional drive system, running with brushless DC motors. A camera which is 2 meters above the field provides vision feedback to identify and control the robots. Robot identification is carried out by recognizing colored patches on top of the robots. Several formation strategies are proposed and implemented in the setup using the holonomic robots we have built.