Forests are vital to our ecosystems, acting as carbon sinks, climate stabilizers, biodiversity centers, and wood sources. Due to their scale, monitoring and managing forests takes a lot of work. Forestry robotics offers the potential for enabling efficient and sustainable foresting practices through automation. Despite increasing interest in this field, the scarcity of robotics datasets and benchmarks in forest environments is hampering progress in this domain. In this paper, we present a real-world, longitudinal dataset for forestry robotics that enables the development and comparison of approaches for various relevant applications, ranging from semantic interpretation to estimating traits relevant to forestry management. The dataset consists of multiple recordings of the same plots in a forest in Switzerland during three different growth periods. We recorded the data with a mobile 3D LiDAR scanning setup. Additionally, we provide semantic annotations of trees, shrubs, and ground, instance-level annotations of trees, as well as more fine-grained annotations of tree stems and crowns. Furthermore, we provide reference field measurements of traits relevant to forestry management for a subset of the trees. Together with the data, we also provide open-source baseline panoptic segmentation and tree trait estimation approaches to enable the community to bootstrap further research and simplify comparisons in this domain.

Robust and accurate localization and mapping of an environment using laser scanners, so-called LiDAR SLAM, is essential to many robotic applications. Early 3D LiDAR SLAM methods often exploited additional information from IMU or GNSS sensors to enhance localization accuracy and mitigate drift. Later, advanced systems further improved the estimation at the cost of a higher runtime and complexity. This paper explores the limits of what can be achieved with a LiDAR-only SLAM approach while following the "Keep It Small and Simple" (KISS) principle. By leveraging this minimalist design principle, our system, KISS-SLAM, achieves state-of-the-art performance in pose accuracy while requiring little to no parameter tuning for deployment across diverse environments, sensors, and motion profiles. We follow best practices in graph-based SLAM and build upon LiDAR odometry to compute the relative motion between scans and construct local maps of the environment. To correct drift, we match local maps and optimize the trajectory in a pose graph optimization step. The experimental results demonstrate that this design achieves competitive performance while reducing complexity and reliance on additional sensor modalities. By prioritizing simplicity, this work provides a new strong baseline for LiDAR-only SLAM and a high-performing starting point for future research. Furthermore, our pipeline builds consistent maps that can be used directly for downstream tasks like navigation. Our open-source system operates faster than the sensor frame rate in all presented datasets and is designed for real-world scenarios.

Accurate localization is crucial for the autonomous operation of mobile robots. Specifically for indoor scenarios, localization algorithms typically rely on a previously generated map. However, many real-world sites like warehouses or healthcare environments violate the underlying assumption that the robot’s surroundings are mainly static. In this paper, we introduce a new dataset plus a benchmark that enables evaluating and comparing indoor localization methods in complex and changing real-world scenarios. While several datasets for indoor scenes exist, only a few combine the long-term localization aspect of repeatedly revisiting the same environment under varying conditions with precise ground truth over multiple rooms. Our dataset comprises various sequences recorded with a wheeled robot covering an office environment. We provide data from two 2D LiDARs, multiple consumer-grade RGB-D cameras, and the robot’s wheel odometry. By densely placing fiducial markers on every room ceiling, we can also provide accurate pose information within a single global frame for the whole environment, estimated through an additional upward-facing camera. We evaluate existing localization algorithms on our data and make the dataset together with a server-based benchmark evaluation publicly available. This facilitates an unbiased evaluation of localization approaches and enables further research on their application in challenging indoor scenarios.

The ability to detect loop closures plays an essential role in any SLAM system. Loop closures allow correcting the drifting pose estimates from a sensor odometry pipeline. In this paper, we address the problem of effectively detecting loop closures in LiDAR SLAM systems in various environments with longer lengths of sequences and agnostic of the scanning pattern of the sensor. While many approaches for loop closures using 3D LiDAR sensors rely on individual scans, we propose the usage of local maps generated from locally consistent odometry estimates. Several recent approaches compute the maximum elevation map on a bird’s eye view projection of point clouds to compute feature descriptors. In contrast, we use a density image bird’s eye view representation, which is robust to viewpoint changes. The utilization of dense local maps allows us to reduce the complexity of features describing these maps, as well as the size of the database required to store these features over a long sequence. This yields a real-time application of our approach for a typical robotic 3D LiDAR sensor. We perform extensive experiments to evaluate our approach against other state-of-the-art approaches and show the benefits of our proposed approach.

Forests play a crucial role in our ecosystems, functioning as carbon sinks, climate stabilizers, biodiversity hubs, and sources of wood. By the very nature of their scale, monitoring and maintaining forests is a challenging task. Robotics in forestry can have the potential for substantial automation toward efficient and sustainable foresting practices. In this paper, we address the problem of automatically producing a forest inventory by exploiting LiDAR data collected by a mobile platform. To construct an inventory, we first extract tree instances from point clouds. Then, we process each instance to extract forestry inventory information. Our approach provides the per-tree geometric trait of "diameter at breast height" together with the individual tree locations in a plot. We validate our results against manual measurements collected by foresters during field trials. Our experiments show strong segmentation and tree trait estimation performance, underlining the potential for automating forestry services. Results furthermore show a superior performance compared to the popular baseline methods used in this domain.

Odometry estimation is crucial for every autonomous system requiring navigation in an unknown environment. In modern mobile robots, 3D LiDAR-inertial systems are often used for this task. By fusing LiDAR scans and IMU measurements, these systems can reduce the accumulated drift caused by sequentially registering individual LiDAR scans and provide a robust pose estimate. Although effective, LiDAR-inertial odometry systems require proper parameter tuning to be deployed. In this paper, we propose LIO-EKF, a tightly-coupled LiDAR-inertial odometry system based on point-to-point registration and the classical extended Kalman filter scheme. We propose an adaptive data association that considers the relative pose uncertainty, the map discretization errors, and the LiDAR noise. In this way, we can substantially reduce the parameters to tune for a given type of environment. The experimental evaluation suggests that the proposed system performs on par with the state-of-the-art LiDAR-inertial odometry pipelines but is significantly faster in computing the odometry. The source code of our implementation is publicly available (https://github.com/YibinWu/LIO-EKF).

Tracking changes in growing plants is important for automating phenotyping and robots managing crops. In this paper, we propose a system that uses a 3D model of plants along crop rows to enable a robotic platform to localize itself even in the presence of heavy changes and deforming the model to adapt the scene description to the new measurements. In particular, we focus on consumer RGB-D cameras due to their cost-effectiveness and ease of deployment on real platforms. Our approach exploits modern deep-learning-based feature descriptors and geometric information to obtain matches between 3D points corresponding to temporally distant sessions. We then use the associations in a non-rigid registration pipeline to obtain the final result, an updated representation of the 3D model that reflects plant changes. Using a standard RGB-D sensor, we validate our approach on a real-world dataset recorded in a glasshouse. We obtain accurate 4D models of the plants and track the plant traits’ evolution over time. We show, through experiments, that our method is applicable to interpolate plant organs’ evolution, a helpful result for phenotypic trait measurement. We see our approach as a relevant step toward 4D reconstruction for robotic agriculture in the wild.

Crop inspection is a critical part of modern agricultural practices that helps farmers assess the current status of a field and then make crop management decisions. Current crop inspection methods are labour-intensive tasks, which makes them rather slow and expensive to apply. In this paper, we exploit recent advancements in implicit mapping to tackle the challenging context of agricultural environments to create dense maps of crop rows with high enough fidelity to be useful for automated crop inspection. Specifically, we map strawberry and sweet pepper crop rows using RGB images captured by a wheeled mobile field robot inside a greenhouse and then use this data to build 3D maps to document the development of plants and fruits. Our Target-Aware Implicit Mapping system (TAIM) uses a SLAM-based pose initialization strategy for robust pose convergence, an efficient information-guided training sample selection framework for faster loss reduction, and focuses on exploiting training samples for fruit regions of the scene, which are critical for crop inspection tasks, to create more accurate maps in less time.

Monitoring the traits of plants and fruits is a fundamental task in horticulture. With accurate measurements, farmers can predict the yield of their crops and use this information for making informed management decisions, and breeders can use it for variety selection. Agricultural robotic applications promise to automate this monitoring task. In this paper, we address the problem of monitoring fruit growth and investigate the matching of fruits recorded in commercial greenhouses at different growth stages based on data recorded from terrestrial laser scanners. This is challenging as fruits appear highly similar, change over time, and are subject to severe occlusions. We first propose a fruit descriptor, which captures the topology of the fruit surroundings to facilitate the matching between different points in time. We capture and describe the relationship between a fruit and its neighbors such that our descriptors are less affected by the growth over time. Furthermore, we define a matching cost function and use an optimal assignment algorithm to match the fruit observations taken in different weeks. The experiments show that our descriptor achieves a high spatio-temporal matching accuracy, which is superior to the commonly used geometric point cloud descriptors.

Agricultural robots have the prospect to enable more efficient and sustainable agricultural production of food, feed, and fiber. Perception of crops and weeds is a central component of agricultural robots that aim to monitor fields and assess the plants as well as their growth stage in an automatic manner. Semantic perception mostly relies on deep learning using supervised approaches, which require time and qualified workers to label fairly large amounts of data. In this paper, we look into the problem of reducing the amount of labels without compromising the final segmentation performance. For robots operating in the field, pre-training networks in a supervised way is already a popular method to reduce the number of required labeled images. We investigate the possibility of pre-training in a self-supervised fashion using data from the target domain. To better exploit this data, we propose a set of domain-specific augmentation strategies. We evaluate our pre-training on semantic segmentation and leaf instance segmentation, two important tasks in our domain. The experimental results suggest that pre-training with domain-specific data paired with our data augmentation strategy leads to superior performance compared to commonly used pre-trainings. Furthermore, the pre-trained networks obtain similar performance to the fully supervised with less labeled data.

Plant phenotyping is a central task in agriculture, as it describes plants' growth stage, development, and other relevant quantities. Robots can help automate this process by accurately estimating plant traits such as the number of leaves, leaf area, and the plant size. In this paper, we address the problem of joint semantic, plant instance, and leaf instance segmentation of crop fields from RGB data. We propose a single convolutional neural network that addresses the three tasks simultaneously, exploiting their underlying hierarchical structure. We introduce task-specific skip connections, which our experimental evaluation proves to be more beneficial than the usual schemes. We also propose a novel automatic post-processing, which explicitly addresses the problem of spatially close instances, common in the agricultural domain because of overlapping leaves. Our architecture simultaneously tackles these problems jointly in the agricultural context. Previous works either focus on plant or leaf segmentation, or do not optimise for semantic segmentation. Results show that our system has superior performance compared to state-of-the-art approaches, while having a reduced number of parameters and is operating at camera frame rate.

Perception is crucial for robots that act in real-world environments, as autonomous systems need to see and understand the world around them to act properly. Panoptic segmentation provides an interpretation of the scene by computing a pixelwise semantic label together with instance IDs. In this paper, we address panoptic segmentation using RGB-D data of indoor scenes. We propose a novel encoder-decoder neural network that processes RGB and depth separately through two encoders. The features of the individual encoders are progressively merged at different resolutions, such that the RGB features are enhanced using complementary depth information. We propose a novel merging approach called ResidualExcite, which reweighs each entry of the feature map according to its importance. With our double-encoder architecture, we are robust to missing cues. In particular, the same model can train and infer on RGB-D, RGB-only, and depth-only input data, without the need to train specialized models. We evaluate our method on publicly available datasets and show that our approach achieves superior results compared to other common approaches for panoptic segmentation.

Our world is non-static, and robots should be able to track its changing geometry. For tracking changes, data asso-ciations between 3D points over time are key. In this paper, we investigate the problem of associating 3D points on plant organs from different mapping runs over time while the plants grow. We achieve a high spatial-temporal matching performance by combining 3D RGB-D SLAM, visual place recognition, and 2D/3D matching exploiting background knowledge. We showcase our approach in a real agricultural glasshouse used to grow sweet peppers, using RGB-D observations from a mobile robot traversing the environment. Our experiments suggest that with our approach, we can robustly make data associations in highly repetitive scenes and under changing geometries caused by plant growth. We see our approach as an important step towards spatial-temporal data association for robotic agriculture.

Simultaneous Localization and Mapping (SLAM) systems are fundamental building blocks for any autonomous robot navigating in unknown environments. The SLAM implementation heavily depends on the sensor modality employed on the mobile platform. For this reason, assumptions on the scene's structure are often made to maximize estimation accuracy. This paper presents a novel direct 3D SLAM pipeline that works independently for RGB-D and LiDAR sensors. Building upon prior work on multi-cue photometric frame-to-frame alignment [4], our proposed approach provides an easy-to-extend and generic SLAM system. Our pipeline requires only minor adaptations within the projection model to handle different sensor modalities. We couple a position tracking system with an appearance-based relocalization mechanism that handles large loop closures. Loop closures are validated by the same direct registration algorithm used for odometry estimation. We present comparative experiments with state-of-the-art approaches on publicly available benchmarks using RGB-D cameras and 3D LiDARs. Our system performs well in heterogeneous datasets compared to other sensor-specific methods while making no assumptions about the environment. Finally, we release an open-source C++ implementation of our system.

Robust localization in a given map is a crucial component of most autonomous robots. In this paper, we address the problem of localizing in an indoor environment that changes and where prominent structures have no correspondence in the map built at a different point in time. To overcome the discrepancy between the map and the observed environment caused by such changes, we exploit human-readable localization cues to assist localization. These cues are readily available in most facilities and can be detected using RGB camera images by utilizing text spotting. We integrate these cues into a Monte Carlo localization framework using a particle filter that operates on 2D LiDAR scans and camera data. By this, we provide a robust localization solution for environments with structural changes and dynamics by humans walking. We evaluate our localization framework on multiple challenging indoor scenarios in an office environment. The experiments suggest that our approach is robust to structural changes and can run on an onboard computer. We release an open source implementation of our approach11https://github.com/PRBonn/tmcl, which uses off-the-shelf text spotting, written in C++ with a ROS wrapper.

Simultaneous Localization and Mapping (SLAM) is considered a mature research field with numerous applications and publicly available open-source systems. Despite this maturity, existing SLAM systems often rely on ad-hoc implementations or are tailored to predefined sensor setups. In this work, we tackle these issues, proposing a novel unified SLAM architecture specifically designed to standardize the SLAM problem and to address heterogeneous sensor configurations. Thanks to its modularity and design patterns, the presented framework is easy to extend, maximizes code reuse and improves computational efficiency. We show in our experiments with a variety of typical sensor configurations that these advantages come without compromising state-of-the-art SLAM performance. The result demonstrates the architecture's relevance for facilitating further research in (multi-sensor) SLAM and its transfer into practical applications.

In this paper, we present a sonar-based navigation system, designed to deploy a fleet of autonomous mobile platforms at a reasonable cost. In educational and hobbyist contexts, a large number of robots is required. By means of classical navigation approaches, every robot should be provided with accurate vision or range sensors. This limits the maximum number of robots in the fleet, due to the unaffordable cost of these sensors. In contrast to that, our system requires a single platform equipped with a higher quality sensor, used to perform calibration and mapping tasks. The rest of the fleet, able to localize and navigate, is equipped solely with low-cost sonars, providing a notable reduction in the overall cost. We achieve this task by presenting a novel calibration procedure to estimate the sonars extrinsic and adapting a classical monte-carlo localization algorithm to the sonar model, focusing on efficiency. We release an open source implementation of the system to the community.