Few-shot object detection is especially interesting for applications with mobile robots and becomes even more challenging when task-related classes are very similar. This work focuses on such a scenario: detecting different types of household and industrial tools. Such tools can be rare and specific and are usually not covered by existing large datasets, except for common ones such as screwdrivers. Additionally, the target classes might change frequently depending on the robot’s missions. Therefore, we propose DE-fine-ViT, a fine-grained few-shot object detection model that does not require fine-tuning. We build our architecture on top of the elaborate DE-ViT model, extending it with specialized components to improve the fine-grained detection capabilities. The user can construct class and part prototypes tailored to the task in an interactive preparation phase. During inference, our proposed reevaluation module leverages the multi-granularity of prototypes for fine-grained class differentiation. We evaluate our model in multiple realistic experiments, including a specifically created fine-grained dataset, demonstrating its efficacy and suitability for scenarios with little data and low inter-class variance.

Mobile robots can perform increasingly impressive feats in controlled environments. Many real applications, though, especially for walking robots, introduce a high degree of unforeseen difficulties, yet require very robust robot operation. In these cases, it is still often not possible to guarantee the needed reliability.We present an approach to utilize unsupervised anomaly detection to implement a fear-based adaptation of robot behavior. This allows robots to automatically and quickly react to any type of unexpected problems. Neither the environment nor the type of disturbance has to be known beforehand, as the system requires only a small amount of baseline data for training, which can be collected in a laboratory environment. Additionally, it can work on arbitrary robot hardware and be integrated in all types of robot control structures.We evaluated our approach in simulation and on state of the art walking robots, ANYmal, Spot and our own six-legged walking robot prototype, in a realistic field test environment in the Tabernas desert in Spain. Our results showcase that we can quickly detect arbitrary problems based on significantly different types of sensor data and decrease robot fall rates in the most extreme scenarios from 56% to 4%. This promises significant increases in robustness for all types of walking robots in highly challenging and previously unknown environments.

Enhancing production efficiency, ensuring consistent quality, and significantly reducing manufacturing costs are core benefits of deploying robotic solutions to manufacturing tasks. Leading to automation of assembly tasks being a focus of robotics research since many years, and many solutions have already been successfully deployed into the industry. However, applying those approaches in a human-robot collaboration (HRC) scenario is challenging. Humans introduce an additional uncertainty factor that prohibits pure offline planning and requires plan adaptation during execution. Intelligent planning approaches are therefore needed that consider human intervention from the start and enable further progress toward flexible HRC assembly systems. This paper presents a novel approach to flexible robotic task planning by utilizing assembly affordances, which are perceived opportunities for actions that a component offers in terms of assembly. Our approach integrates affordance-based reasoning within a semantic planning framework to allow online refinement of assembly plans to be used in fully automated scenarios as well as HRC assembly scenarios. The paper presents the developed concept and framework and evaluates it based on the success rate on assemblies of different complexities.

Deciding where to go next is a challenging task for humans. However, for robots in unknown environments, this becomes even more demanding. In planetary explorations, the robots are continuously challenged with the task of exploring novel areas, yet so far, humans decide for the robots where to go. Even then, prioritizing the next target based on previous knowledge is complex. In our proposed work, the robot utilizes data about its surroundings from drone or satellite images. Alternatively, a volumetric representation can be reduced to form a suitable input. From the input, tiles are selected and embedded by different autoencoder variants. The robot can select the most promising next exploration goal through the distance in the embedding to the previous samples. In this work, a variational autoencoder, a Wasserstein autoencoder, and a spherical autoencoder are evaluated against each other. The latter two variants yield a high information gain when evaluated on satellite data from the Netherlands. Additionally, the framework was employed on data from an analog mission in the Tabernas desert. Through the framework, the robots get an understanding of which goals yield the most information gain and, therefore, can quickly improve their knowledge about their surroundings.

The use of mobile robots has become increasingly common in multiple areas of daily life. To increase their autonomy for performing various tasks, efficient navigation skills are essential. The most crucial component of such navigation is the ability to calculate a global path between two points. The global path planning problem for mobile robots is typically limited to two-dimensional environments, in which the environment is projected onto a planar surface. While this approach works well in structured environments like industrial settings, it may not be suitable for all applications of mobile robots. With modern walking robots, capable of navigating complex terrain, more advanced path planning approaches are necessary. This work proposes a path-planning approach that utilizes the entire three-dimensional space, allowing for navigation in even the most challenging terrain. The central idea is to extend a traditional A* path planner to work directly on a fast volumetric map structure to generate optimal paths through the environment. Multiple optimizations and adjustments are introduced to improve the algorithm’s performance. By applying morphology operators to sparse maps, sensor inaccuracies during the map construction are mitigated. Additionally, adjustments are made to handle the added complexity introduced by the extra search space dimension and to comply with the limitations of autonomous walking robots. This is paired with an efficient caching strategy to enhance the overall path-planning speed. The capability of the path planning approach is evaluated using both artificial and real-world maps. The results demonstrate that this approach shows great potential for enabling mobile ground robots to autonomously navigate even the most demanding terrains utilizing the entire three-dimensional space.

Exploring unknown environments, such as caves or planetary surfaces, requires a quick understanding of the surroundings. Beforehand, only aerial footage from satellites or images from previous missions might be available. The proposed ensemble based anomaly detection framework utilizes previously gained knowledge and incorporates it with insights gained during the mission. The modular system consists of different networks which are combined to determine anomalies in the current surroundings. By utilizing data from other missions, simulations or aerial photos, a precise anomaly detection can be achieved at the start of a mission. The system can further be improved by training new networks during the mission, which can be incorporated into the ensemble at runtime. This allows for synchronous execution of mission and training of models on a base station. The proposed system is tested and evaluated on an ANYmal C walking robot in different scenarios, however the approach is applicable for different kinds of mobile robots. The results show a clear improvement of ensembles compared to individual networks, while keeping a small memory footprint and low inference time on the mobile system.

Complex robots in challenging scenarios require constant monitoring of their state and adaptation of their behavior to ensure robustness, reliability and longevity. While known possible errors can be specifically surveilled, other prob-lems can be fully unforeseen, requiring detection systems able to identify novel faults. We detect possible faults as anomalies on various internal sensor data, utilizing unsupervised learning techniques. A bidirectional Wasserstein GAN approach for anomaly detection on multivariate, highly dependent time-series data is implemented and trained on a small amount of non-anomalous robot sensor data. This model is then used for inference on the on-board hardware of a robot without parallel processing units. We evaluate multiple variants of the architecture using manually introduced anomalies in the form of different weights attached to the robot's legs. Overall we are able to show that RoBiGAN is able to consistently detect and localize small anomalies in an online scenario, with little to no robot specific modeling needed.

Field and service robots that do not understand the hazards in their environment limit their potential by acting overly careful or navigating into potentially dangerous areas. We present an extended modular mapping framework which allows to model different types of hazards from a multitude of inputs. The proposed approach is generalized for storage of arbitrary data, therefore not limiting the usage to one use case. Furthermore the framework allows the fusion of risks to calculate the overall risk for an individual robot from its surroundings. The system was tested with LAURON V, a hexapod designed for walking over rough and hazardous terrain.