Though larger vessels may be well-equipped to deal with wavy conditions, smaller vessels are often more susceptible to disturbances. This paper explores the development of a nonlinear model predictive control (NMPC) system for Uncrewed Surface Vessels (USVs) in wavy conditions to minimize average roll. The NMPC is based on a prediction method that uses information about the vessel’s dynamics and an assumed wave model. This method is able to mitigate the roll of an under-actuated USV in a variety of conditions by adjusting the weights of the cost function. The results show a reduction of 39 % of average roll with a tuned controller in conditions with 1.75-metre sinusoidal waves. A general and intuitive tuning strategy is established. This preliminary work is a proof of concept which sets the stage for the leveraging of wave prediction methodologies to perform planning and control in real time for USVs in real-world scenarios and field trials.

Path planning and localization in low-light and inclement weather conditions are critical problems facing autonomous vehicle systems. Our proposed method applies a single modality, millimetre-wave radar perception system for the detection of roadside retro-reflectors. Radar-based perception tasks can be challenging to perform due to the sparse and noisy nature of radar data. We propose the use of an unsupervised learning approach for filtering radar point clouds through Density-Based Spatial Clustering of Applications with Noise (DBSCAN). The DBSCAN algorithm segments retro-reflector points from noise points, thus providing the autonomous vehicle with a predicted path for the road ahead. We tested the approach via indoor experiments that make use of Continental's ARS 408 radar, a mobile Husky A2000 robot, and a Vicon motion capture system for ground truth validation. The experimental results of the proposed system demonstrated a classification accuracy of 84.13 % and F1 score of 83.71 %.

Mobile robots are well suited for environmental surveys because they can travel to any area of interest and react to observations without the need for pre-existing infrastructure or significant setup time. However, vehicle motion constraints limit where and when measurements occur. This is challenging for a single vehicle observing a time-varying phenomenon, such as coastal waves, but the ability to generate a spatiotemporal map would have immediate scientific and engineering applications. In this paper, an uncrewed surface vessel (USV) was used to measure waves on the coast of Lake Ontario, Canada. Data were collected from a low-cost inertial measurement system onboard the USV and processed in an offline Gaussian process regression (GPR) workflow to create a spatiotemporal wave model. Frequency analysis of raw sensor data was used to best select and design kernel functions, and to initialize hyperparameters. The relative speed of the waves limited the ability to make complete wave reconstructions, but GPR captured the dominant periodic components of the waves despite irregularities in the signals. After optimization, the hyperparameters indicate a dominant signal with a wave period of 0.87$\mathbf{s}$, which concurs with ground truth estimates.

This paper proposes the Real-Time Fast Marching Tree (RT-FMT), a real-time planning algorithm that features local and global path generation, multiple-query planning, and dynamic obstacle avoidance. During the search, RT-FMT quickly looks for the global solution and, in the meantime, generates local paths that can be used by the robot to start execution faster. In addition, our algorithm constantly rewires the tree to keep branches from forming inside the dynamic obstacles and to maintain the tree root near the robot, which allows the tree to be reused multiple times for different goals. Our algorithm is based on the planners Fast Marching Tree (FMT*) and Real-time Rapidly-Exploring Random Tree (RT-RRT*). We show via simulations that RT-FMT outperforms RT- RRT* in both execution cost and arrival time, in most cases. Moreover, we also demonstrate via simulation that it is worthwhile taking the local path before the global path is available in order to reduce arrival time, even though there is a small possibility of taking an inferior path.

3D hand pose estimation (HPE) is the process of locating the joints of the hand in 3D from any visual input. HPE has recently received an increased amount of attention due to its key role in a variety of human-computer interaction applications. Recent HPE methods have demonstrated the advantages of employing videos or multi-view images, allowing for more robust HPE systems. Accordingly, in this study, we propose a new method to perform Sequential learning with Transformer for Hand Pose (SeTHPose) estimation. Our SeTHPose pipeline begins by extracting visual embeddings from individual hand images. We then use a transformer encoder to learn the sequential context along time or viewing angles and generate accurate 2D hand joint locations. Then, a graph convolutional neural network with a U-Net configuration is used to convert the 2D hand joint locations to 3D poses. Our experiments show that SeTHPose performs well on both hand sequence varieties, temporal and angular. Also, SeTHPose outperforms other methods in the field to achieve new state-of-the-art results on two public available sequential datasets, STB and MuViHand.

Spatiotemporal maps are data-driven estimates of time changing phenomena. For environmental science, rather than collect data from an array of static sensors, a mobile sensor platform could reduce setup time and cost, maintain flexibility to be deployed to any area of interest, and provide active feedback during observations. While promising, mapping is challenging with mobile sensors because vehicle constraints limit not only where, but also when observations can be made. By assuming spatial and temporal correlations in the data through kernel functions, this paper uses Gaussian process regression (GPR) to generate a maximum likelihood estimate of the phenomenon while also tracking the estimate uncertainty. Spatiotemporal mapping by GPR is simulated for a single fixed-path mobile robot observing a latent spatiotemporal scalar field. The learned spatiotemporal map captures the structure of the latent scalar field with the largest uncertainties in areas the robot never visited.

We introduce a self-supervised method for systematically choosing traversable terrain while autonomously navigating a vehicle to a goal position in an unknown off-road environment. Leveraging the color discriminant bias of off-road terrain types, and using images from a vehicle-mounted camera, we employ a viewpoint transformation that maintains the spatial layout of the terrain to cluster terrain types by color and register corresponding traversability features to guide future navigation decisions. As it navigates, our algorithm also generates training images for use in contemporary end-to-end navigation schemes. Our test results demonstrate the advantages of our approach over classical near-to-far approaches in off-road environments with unknown traversability characteristics, and highlight its fit to supervised semantic segmentation schemes that require foreknowledge of traversability characteristics for labeling, which are limited by insufficient data and suffer pixel-level class imbalance. We detail the techniques for clustering, feature registration, path planning and navigation; and demonstrate the method. Finally, we study the effectiveness of non-discretionary self-supervised data labeling.

This paper presents a learning-based Model Predictive Control (MPC) methodology incorporating nonlinear predictions with robotics applications in mind. In particular, MPC is combined with feedback linearization for computational efficiency and Gaussian Process Regression (GPR) is used to model unknown system dynamics and nonlinearities. In this method, MPC predicts future states by leveraging a GPR model and optimizes a sequence of inputs over feedback linearized states. The controller was tested in simulation by using a two-link planar robot in the presence of model uncertainty. With respect to trajectory-tracking error, the proposed controller outperformed a conventional Proportional-Derivative Inverse Dynamics controller and a GPR-augmented version. Although a fully nonlinear MPC formulation achieved slightly better performance, the proposed controller had an average control calculation time that was 82× faster.

A novel method is presented to robustly and efficiently register two partially overlapping point clouds. Following segmentation, the regions are represented as implicit quadric surfaces using polynomials of degree two in three variables. The registration establishes correspondences of virtual interest points, which do not exist in the original point cloud data, and are defined by the intersection of three implicit quadric surfaces extracted from the point cloud regions. Implicit quadric surfaces exist in abundance in both natural and architectural scenes, and can be used to identify stable regions in the data, which in turn leads to repeatable virtual interest points. Large regions in a point cloud can be represented by a few implicit surfaces, which reduces the computational cost of registration and also makes the algorithm robust to noise and data density variations. Experiments were performed on seven data sets from various sensors. The proposed method outperformed most of the feature based registration and non-feature based registration methods for computational efficiency and convergence.

This paper presents a path following method for autonomous wheeled vehicles that combines iterative learning control (ILC) with nonlinear feedback linearization (FBL) to provide anticipatory control action based on stored path following errors over repeated driving trials. By implementing ILC in a fully feedback-linearized space, control corrections are applied to a transformed input, thus allowing for a single back-computation to the nonlinear vehicle's control input. Hence, the approach is comparatively easy to implement and also computationally inexpensive. We first outline the mathematical formulation for this control method and then describe field results from tests conducted by using an industrial-scale wheeled underground mining vehicle in a representative environment to demonstrate effectiveness.

An axis map (AM) represents the orientations of planar surfaces in an environment and is void of positional information. Three-dimensional axis mapping (3DAM) is a graph-based optimization algorithm that generates an AM, while carefully addressing the parameterization of axes (i.e., normal vectors). 3DAM exploits the lack of positional information to form a densely connected graph. This paper provides a detailed description of the parameterization of axes, an outline of the 3DAM algorithm, as well as outdoor experimental results showing the usefulness of AMs as a tool for mobile autonomous geotechnical mapping.

This paper introduces two-dimensional axis mapping, which estimates axis maps (AMs) based on LiDAR measurements. An AM describes the dominant orientations of surfaces in an environment and is void of positional information. As a consequence of the directional nature of the map, there are significant differences compared with traditional mapping algorithms. Experimental results are presented from simulated, indoor, outdoor, and underground environments. One major application of AMs is for heading estimation. An example of this application is shown to effectively bound the growth of heading error in both an indoor and an outdoor environment.

This paper introduces an accurate and inexpensive method for localizing a calibrated monocular camera in 3D indoor environments. The objective of this work is to localize in 6 degrees-of-freedom (6 DOF) in the presence of a 3D map that contains 3D point clouds co-registered with intensity information. This is done by solving the Perspective-n-Point (PnP) problem to accurately compute the camera location in 6 DOF. An efficient data structure is used to store a large set of point clouds co-registered with intensity information, image features, and transformations between the images. This data structure, referred to as the feature database, is implemented such that it retrieves a match for a query image efficiently. Thus the overall process of localization in 6 DOF becomes a real-time process with high efficiency and accuracy. Our technique was tested with two ground truth data sets of indoor environments, an office and a laboratory. The experimental results show the accuracy and the efficiency of our technique, with an average localization error of less than 10 mm from the ground truth in both environments. In addition, localization results on query images obtained using two different cameras in four different environments are presented. This demonstrates that any type of monocular camera may be used during localization, as long as a sufficient number of environmental features can be extracted from the query images.

Although passive sensors are widely used for many mobile robotics applications that perform mapping and localization functions, there are many environments (e.g., mining and planetary) where active sensors are more practical. However, at present, most 3D SLAM algorithms that do use LiDAR and/or time-of-flight (ToF) sensors exploit only range and bearing information associated with these measurements, but not intensity information. This paper presents a new approach that attempts to explicitly incorporate an intensity model as part of a sparse bundle adjustment (SBA) estimation problem. An observability analysis shows that a solution exists, and simulation results verify its potential utility.

This paper presents a solution to part of the problem of making robotic or semi-robotic digging equipment less dependant on human supervision. A method is described for identifying rocks of a certain size that may affect digging efficiency or require special handling. The process involves three main steps. First, by using range and intensity data from a time-of-flight (TOF) camera, a feature descriptor is used to rank points and separate regions surrounding high scoring points. This allows a wide range of rocks to be recognized because features can represent a whole or just part of a rock. Second, these points are filtered to extract only points thought to belong to the large object. Finally, a check is carried out to verify that the resultant point cloud actually represents a rock. Results are presented from field testing on piles of fragmented rock.

This paper presents the results and subsequent analysis of full-scale excavation experiments aimed at developing a practical understanding of how actuator forces evolve during excavation and how they relate to the interactions that occur between an excavator's end-effector and its environment. Our focus is on the excavation problem for fragmented rock, as is common in mining and construction applications. Based on an analysis of the experimental data, an example admittance-type autonomous excavation controller is postulated.