Collisions on inland waters frequently occur due to the absence of clearly defined navigation guidance. To prevent such accidents, optical sensors such as stereo cameras can be employed to detect obstacles at an early stage, enabling timely warnings to vessel operators or the planning of collision avoidance trajectories. In this context, the paper presents a neural network for object detection on inland waters, designed to address the specific challenges of waterborne vehicles, including strong ego-motion and contextual cues like the shoreline appearing in the background. The proposed network leverages a plane sweep approach to integrate multiple views of a scene and predict object locations in bird’s-eye view (BEV) coordinates. Its ability to incorporate more than two camera perspectives is demonstrated using the KITTI dataset. On real-world inland water data, the network is evaluated against both a traditional maritime stereo-based method and a learning-based stereo detector from autonomous driving, showing consistently higher mean average precision. The code is available at https://github.com/dionysos4/IWOD.

In extended object tracking, random matrices are commonly used to filter the mean and covariance matrix from measurement data. However, the relation from mean and covariance matrix to the extension parameters can become challenging when a lidar sensor is used. To address this, we propose virtual measurement models to estimate those parameters iteratively by adapting them, until the statistical moments of the measurements they would cause, match the random matrix result. While previous work has focused on 2D shapes, this paper extends the methodology to encompass 3D shapes such as cones, ellipsoids and rectangular cuboids. Additionally, we introduce a classification method based on Chamfer distances for identifying the best-fitting shape when the object’s shape is unknown. Our approach is evaluated through simulation studies and with real lidar data from maritime scenarios. The results indicate that a cone is the best representation for sailing boats, while ellipsoids are optimal for motorboats.

Autonomous navigation on inland waters requires an accurate understanding of the environment in order to react to possible obstacles. Deep learning is a promising technique to detect obstacles robustly. However, supervised deep learning models require large data-sets to adjust their weights and to generalize to unseen data. Therefore, we equipped our research vessel with a laser scanner and a stereo camera to record a novel obstacle detection data-set for inland waters. We annotated 1974 stereo images and lidar point clouds with 3d bounding boxes. Furthermore, we provide an initial approach and a suitable metric2to compare the results on the test data-set. The data-set is publicly available3and seeks to make a contribution towards increasing the safety on inland waters.

Random matrices are used to filter the center of gravity (CoG) and the covariance matrix of measurements. However, these quantities do not always correspond directly to the position and the extent of the object, e.g. when a lidar sensor is used.In this paper, we propose a Gaussian processes regression model (GPRM) to predict the position and extension of the object from the filtered CoG and covariance matrix of the measurements. Training data for the GPRM are generated by a sampling method and a virtual measurement model (VMM). The VMM is a function that generates artificial measurements using ray tracing and allows us to obtain the CoG and covariance matrix that any object would cause. This enables the GPRM to be trained without real data but still be applied to real data due to the precise modeling in the VMM. The results show an accurate extension estimation as long as the reality behaves like the modeling and e.g. lidar measurements only occur on the side facing the sensor.

Motion estimation is an essential element for autonomous vessels. It is used e.g. for lidar motion compensation as well as mapping and detection tasks in a maritime environment. Because the use of gyroscopes is not reliable and a high performance inertial measurement unit is quite expensive, we present an approach for visual pitch and roll estimation that utilizes a convolutional neural network for water segmentation, a stereo system for reconstruction and simple geometry to estimate pitch and roll. The algorithm is validated on a novel, publicly available dataset22https://git.ios.htwg-konstanz.de/dgriesse/constance_orientation_dataset/archive/main/constance_orientation_dataset-main.zip recorded at Lake Constance. Our experiments show that the pitch and roll estimator provides accurate results in comparison to an Xsens IMU sensor. We can further improve the pitch and roll estimation by sensor fusion with a gyroscope. The algorithm is available in its implementation as a ROS node33https://github.com/dionysos4/water_surface_detector.