Weed control is important in agriculture and gardening but typically employed manual solutions are both time and labour consuming. By combining advanced developments in digitalization and AI (artificial intelligence), this paper presents two autonomous weed-control related processes: i) for the mapping of rumex weed in an unexplored agricultural field and ii) for the surveillance of the marked herbs. The processes employ a mobile robot equipped with a LIDAR sensor (mainly used for navigation and mapping) and a front stereo camera. A YOLOv8 object detector which is specifically trained for rumex detection is used to locate rumex plants in the camera image. For the first application of mapping, the weeds are marked on the map of the field while for the second task of surveillance, the Ant Colony Optimization looks for an efficient route connecting all those marked weeds which allows the robot to revisit and monitor them. The proposed processes are validated under both real experiments and simulation in realistic Gazebo environments.

Navigation in agricultural fields imposes various constraints on manoeuvrability, which can be tackled by using four-wheel steering (4WS) vehicles which are capable of switching between multiple steering mechanisms with distinct kinematic properties. For example, parallel positive steering (PPS) with four wheels in parallel to each other can maintain the vehicle’s heading when moving along a curve. Symmetric negative steering (SNS) with two wheels on each side sharing the same steering angle can turn with a small radius. This paper presents a controller capable of selecting and switching between the two aforementioned modes autonomously for better trajectory tracking performance with special heading requirements for agricultural applications. The controller is implemented as a Model Predictive Control (MPC) controller formulated as a mixed-integer quadratic programming (MIQP) problem for the 4WS vehicle. Practical constraints, such as limits on wheel velocities, steering angles and their rate-of-changes are taken into account. A Python implementation confirms the real-time execution capability of the controller and simulation results highlight its effectiveness.

4D ultrasound (US) imaging is a promising imaging modality for diagnosis as well as therapy guidance. It provides volumetric images of soft-tissue structures in real-time without harming the patient with ionizing radiation. Soft tissue targets in the liver are affected by breathing-induced movements resulting in high dimensional motion patterns including deformations affecting the target appearance. Thus, using 4D US for target tracking in radiation therapy to enhance the treatment accuracy is promising but challenging. In this study, a novel approach for target tracking in 4D US is proposed based on predicting the appearance of the target for the next US frame. A spatio-temporal autoencoder as well as a recurrent neural network are implemented and trained based on a labeled 4D US data set with a mean spatial resolution of$1.0\times 0.7\times 1.5$mm3for predicting volumetric target patches and target locations. Four template matching-based tracking algorithms are implemented and evaluated in terms of tracking accuracy. The results indicate that using predicted targets for template matching reduce the tracking error by 62 %, on average. By using a naive template matching algorithm, a mean tracking error of$7.55\pm 12.31$mm was measured. In contrast, using predicted targets, a mean tracking error of$2.85\pm 2.04$mm was determined. Since the proposed spatio-temporal autoencoder is independent of the tracking algorithm, it can be combined with any tracking algorithm, making it a promising approach for improving target tracking accuracy.

For training deep neural networks, large data sets are required. Especially in the medical 3D ultrasound (US) image domain, the amount of available data is limited. A common method to enlarge small data sets is using data augmentation. However, simple geometric augmentation techniques like rotation or sheering can lead to unrealistic US images. In this study, a novel structure-dependent deformable augmentation method for 3D US patches is proposed. The approach is based on learning realistic motion patterns from 3D liver US images using a conditional variational auto encoder (CVAE) where the condition represents the anatomical structure type. The CVAE augmentation performance is compared to a baseline variational auto encoder (VAE). It is shown that the CVAE generates deformable augmentations 18.5 percentage points more similar to realistic deformations than the VAE. Furthermore, the method is evaluated in an expert study where experienced radiologists were asked to rate the realism of US patches. The results show that the experts could not distinguish between CVAE augmented and original US patches. Applying the proposed method in a target detection application improved the performance of a neural network by 41 %. The proposed augmentation method is able to apply realistic deformation to 3D US patches which can be used to enlarge a small data set for usage in deep learning applications.

In this paper, we present a new approach to calculate the B-spline-to-Bézler conversion matrix which converts the control points of a uniform B-spline curve into the control points of an equivalent Bézier curve. It takes into account the order of the curve intervals and hence can provide the conversion of the whole curve at once. The algorithm is implemented so that the computation time only increases proportionally with the number of control points until a constant value before getting saturated. Applications include but are not limited to the efficient usage in different optimal path planning algorithms for navigation in 2D non-convex polytopic region as being presented.

This paper firstly presents our optimal path planning algorithm within a$2\mathrm{D}$non-convex, polytopic region defined as a sequence of connected convex polytopes. The path is a B-spline curve but being parametrized with its equivalent Bézier representation. By doing this, the local convexity bound of each curve's interval is significantly tighter. Thus, it allows many more possibilities for constraining the entire curve to remain inside the region by using only linear constraints on the control points of the curve. We further guarantee the existence of the valid path by pointing out an algebraic solution. We integrate the algorithm, together with our previously published results, into the Navigation with polytopes toolbox which can be used as a global path planner, compatible with ROS navigation tools. It provides a framework for constructing a polytope map from a standard occupancy gridmap, searching for an appropriate sequence of connected polytopes and finally, planning a minimal-length path with different options on B-spline or Bézier parametrizations. The validation and comparison with existing methods are done using gridmaps collected under Gazebo simulations and real experiments.

We propose a 2D path planning algorithm in a non-convex workspace defined as a sequence of connected convex polytopes. The reference path is parameterized as a B-spline curve, which is guaranteed to entirely remain within the workspace by exploiting the local convexity property and by formulating linear constraints on the control points of the B-spline. The novelties of the paper lie in the use of the equivalent Bézier representation of the B-spline curve, which significantly reduces the conservatism in the local convexity bound and in the integration of these constraints into a convex quadratic optimization problem, which minimizes the curve length. The algorithm is successfully validated in both simulations and experiments, by providing obstacle-free reference paths on real occupancy grid maps obtained from the laser scan data of a mobile robot platform.

Tools for automatic image analysis are gaining importance in the clinical workflow, ranging from time-saving tools in diagnostics to real-time methods in image-guided interventions. Over the last years, ultrasound (US) imaging has become a promising modality for image guidance due to its ability to provide volumetric images of soft tissue in real-time without using ionizing radiation. One key challenge in automatic US image analysis is the identification of suitable features to describe the image or regions within, e.g. for recognition, alignment or tracking tasks. In recent years, features that were learned data-drivenly provided promising results. Even though these approaches outperformed hand-crafted feature extractors in many applications, there is still a lack of feature learning for local description of three-dimensional US (3DUS) images. In this work, we present a completely data-driven feature learning approach for 3DUS images for usage in target tracking. To this end, we use a 3D convolutional autoencoder (AE) with a custom loss function to encode 3DUS image patches into a compact latent space that serves as a general feature description. For evaluation, we trained and tested the proposed architecture on 3DUS images of the liver and prostate of five different subjects and assessed the similarity between the decoded patches and the original ones. Subject-and organ-specific as well as general AEs are trained and evaluated. Specific AEs could reconstruct patches with a mean Normalized Cross Correlation of 0.85 and 0.81 at maximum in liver and prostate, respectively. It can also be shown that the AEs are transferable across subjects and organs, with a small accuracy decrease to 0.83 and 0.81 (liver, prostate) for general AEs. In addition, a first tracking study was performed to show feasibility of tracking in latent space. In this work, we could show that it is possible to train an AE that is transferable across two target regions and several subjects. Hence, convolutional AEs present a promising approach for creating a general feature extractor for 3DUS.

We present a novel framework for rigid point cloud registration. Our approach is based on the principles of mechanics and thermodynamics. We solve the registration problem by assuming point clouds as rigid bodies consisting of particles. Forces can be applied between both particle systems so that they attract or repel each other. These forces are used to cause rigid-body motion of one particle system toward the other, until both are aligned. The framework supports physics-based registration processes with arbitrary driving forces, depending on the desired behaviour. Additionally, the approach handles feature-enhanced point clouds, e.g., by colours or intensity values. Our framework is freely accessible for download. In contrast to already existing algorithms, our contribution is to precisely register high-resolution point clouds with nearly constant computational effort and without the need for pre-processing, sub-sampling or pre-alignment. At the same time, the quality is up to 28 percent higher than for state-of-the-art algorithms and up to 49 percent higher when considering feature-enhanced point clouds. Even in the presence of noise, our registration approach is one of the most robust, on par with state-of-the-art implementations.

In cranial radiotherapy highly accurate tumor localization is required to spare healthy tissue while delivering sufficient dose to the target. Here, marker-less optical tracking can support immobilization devices and monitor head motion. However, recent studies have shown that the spatial registration of the optically generated point cloud to a reference is not robust enough. It may suffer from misalignments due to local similarities and changes along the deformable surface geometry. Irrespective from the registration algorithm, we propose to support the spatial information by tissue thickness patterns across the surface. These patterns are predicted by Gaussian Processes which process near infrared optical backscatter from the skin. We first demonstrate for a cohort of 30 volunteers that the tissue thickness can be determined with errors of less than 0.22mm on average. We found high prediction accuracy irrespective of gender, age or skin type. Second, we show that the robustness of a standard iterative closest point (ICP) algorithm can be improved when exploiting cutaneous patterns to identify point-to-point correspondences. On average, tissue thickness support outperformed the standard procedure for every subject and was capable of pushing 90% of all misalignments exceeding a registration error of 1mm below that threshold.

Marker-less tracking provides a non-invasive as well as comfortable approach to compensate for head motion in high precision radiotherapy. However, it suffers from a lack of point-to-point correspondences, typically requiring characteristic spatial landmarks to match point clouds. In this study, we show that cutaneous and subcutaneous structures can be uncovered using an 850 nm laser setup. For three subjects, we compare features extracted from camera images with MR scans serving as an anatomical ground truth. The results confirm the validity of the optically detected structures. The negative correlation between skin thickness and reflected light energy is likewise predicted by Monte-Carlo simulations and can be used to improve spatial point cloud matching. Tissue thickness and its facial structure can be predicted with submillimeter accuracy using a Support Vector regression machine. In addition, the optical measurements reveal the location of vessels that are not immediately visible in the MR scan. These promising findings highly encourage its application for a marker-less tracking system.