Categorical Distributional Reinforcement Learning (CDRL) has demonstrated superior sample efficiency in learning complex tasks compared to conventional Reinforcement Learning (RL) approaches. However, the practical application of CDRL is encumbered by challenging projection steps, detailed parameter tuning, and domain knowledge. This paper addresses these challenges by introducing a pioneering Continuous Distributional Model-Free RL algorithm tailored for continuous action spaces. The proposed algorithm simplifies the implementation of distributional RL, adopting an actor-critic architecture wherein the critic outputs a continuous probability distribution. Additionally, we propose an ensemble of multiple critics fused through a Kalman fusion mechanism to mitigate overestimation bias. Through a series of experiments, we validate that our proposed method provides a sample-efficient solution for executing complex continuous-control tasks.

While autonomous racing performance in Time-Trial scenarios has seen significant progress and development, autonomous wheel-to-wheel racing and overtaking are still severely limited. These limitations are particularly apparent in real-life driving scenarios where state-of-the-art algorithms struggle to safely or reliably complete overtaking manoeuvres. This is important, as reliable navigation around other vehicles is vital for safe autonomous wheel-to-wheel racing. The F1Tenth Competition provides a useful opportunity for developing wheel-to-wheel racing algorithms on a standardised physical platform. The competition format makes it possible to evaluate overtaking and wheel-to-wheel racing algorithms against the state-of-the-art. This research presents a novel racing and overtaking agent capable of learning to reliably navigate a track and overtake opponents in both simulation and reality. The agent was deployed on an F1Tenth vehicle and competed against opponents running varying competitive algorithms in the real world. The results demonstrate that the agent’s training against opponents enables deliberate overtaking behaviours with an overtaking rate of 87% compared 56% for an agent trained just to race.

Monocular depth estimation is a rudimentary problem for robotic perception systems and downstream applications. However, depth estimation from a single image is an inherently ill-posed problem due to data loss related to projection from 3D to 2D. Recent studies address the discrepancy between camera parameters by using learning-based methods and unifying the camera model to canonical camera space or bipolar representations, thus addressing the problem of training a metric depth model over different datasets with different camera parameters. In addition, the previous study, OrchardDepth, introduced the sparse-dense depth consistency loss function to learn the dense depth distribution through the city autonomous driving scene to improve model performance in the orchard. Instead of enforcing strict consistency between the sparse and dense depth, this work introduced the KL divergence to encourage the network to adapt to the depth distributions of different sensors and penalize deviations from reliable regions while tolerating errors in unreliable areas. Furthermore, we further enhance the depth consistency loss by integrating bins into the supervised discretised depth distribution. This method significantly improves the robustness and performance of our previous method. In addition, it improves the absolute relative error in the orchard dataset by 17.3% and 16.2% in contrast to SILog Loss and OrchardDepth baseline, respectively. Thus enhancing the new training paradigm for depth estimation in the orchard scene.

Reinforcement Learning (RL) has been widely used to solve tasks where the environment consistently provides a dense reward value. However, in real-world scenarios, rewards can often be poorly defined or sparse. Auxiliary signals are indispensable for discovering efficient exploration strategies and aiding the learning process. In this work, inspired by intrinsic motivation theory, we postulate that the intrinsic stimuli of novelty and surprise can assist in improving exploration in complex, sparsely rewarded environments. We introduce a novel sample-efficient method able to learn directly from pixels, an image-based extension of TD3 with an autoencoder called NaSA-TD3. The experiments demonstrate that NaSA-TD3 is easy to train and an efficient method for tackling complex continuous-control robotic tasks, both in simulated environments and real-world settings. NaSA-TD3 outperforms existing state-of-the-art RL image-based methods in terms of final performance without requiring pre-trained models or human demonstrations.

Apple fruitlet thinning is critical in cultivating high-quality apples, requiring an expert workforce to manage the orchard. The thinning process requires precise mapping of fruitlet clusters across the tree branches to manage the desired load for each tree. This paper presents Archie Snr, which was developed to autonomously assess the current load of the tree and thin the excess apples as an expert thinner would. The platform has been extensively evaluated in a real-world commercial orchard. The results show the platform can generate an average load count accuracy of 82.1% with a recall of 93.3%. The system was then able to successfully thin 66.14% of the fruitlets from the canopy.

Cane pruning grapevines is a complex manual task requiring expert vine assessment to determine which canes to prune. This paper presents Archie Jnr, which was developed to autonomously assess the structure of the vine and prune the lower-quality canes as an expert pruner would. The platform has been extensively evaluated in a real-world commercial vineyard using a three-cane pruning method. The results show the effectiveness of the vision system for generating accurate assessments of a vine’s canes. The platform is also shown to be capable of successfully pruning 71.1% of the 311 total canes that required pruning across 25 vines.

Model Free Reinforcement Learning (MFRL) has shown significant promise for learning dexterous robotic manipulation tasks, at least in simulation. However, the high number of samples, as well as the long training times, prevent MFRL from scaling to complex real-world tasks. Model- Based Reinforcement Learning (MBRL) emerges as a potential solution that, in theory, can improve the data efficiency of MFRL approaches. This could drastically reduce the training time of MFRL, and increase the application of RL for real- world robotic tasks. This article presents a study on the feasibility of using the state-of-the-art MBRL to improve the training time for two real-world dexterous manipulation tasks. The evaluation is conducted on a real low-cost robot gripper where the predictive model and the control policy are learned from scratch. The results indicate that MBRL is capable of learning accurate models of the world, but does not show clear improvements in learning the control policy in the real world as prior literature suggests should be expected.

Aotearoa New Zealand has a strong and growing apple industry but struggles to access workers to complete skilled, seasonal tasks such as thinning. To ensure effective thinning and make informed decisions on a per-tree basis, it is crucial to accurately measure the crop load of individual apple trees. However, this task poses challenges due to the dense foliage that hides the fruitlets within the tree structure. In this paper, we introduce the vision system of an automated apple fruitlet thinning robot, developed to tackle the labor shortage issue. This paper presents the initial design, implementation, and evaluation specifics of the system. The platform straddles the 3.4 m tall 2D apple canopy structures to create an accurate map of the fruitlets on each tree. We show that this platform can measure the fruitlet load on an apple tree by scanning through both sides of the branch. The requirement of an overarching platform was justified since two-sided scans had a higher counting accuracy of 81.17% than one-sided scans at 73.7%. The system was also demonstrated to produce size estimates within 5.9% RMSE of their true size.

This paper presents an emotion inspired adaptive path planning approach for autonomous robotic navigation. Ideally a robotic navigation system should adapt its path planning and behaviour to overcome a variety of obstacles within an environment, without the need for single location planning approaches. Emotional analogies are appealing as they enable general planning, but require hard coding of `emotions'. Humans have a bias on what is an emotion, e.g. fear, which can adversely affect performance. We aim to provide the robot with the generalising ability of emotion without the pre-specifying bias. Inspired by theories on `emotion', the system presented utilises a Learning Classifier System (LCS) to learn a `bow-tie' structure of emotional reinforcers to intermediary emotion categories to a behavioural modifier that adapts the robot's navigation behaviour. The emotional states are not pre-set and are judged post learning based on the learned behaviour. The bow-tie creates a simple compact set of rules to adapt a robot's behaviour to better navigate its environment. The emotion system was verified on a state-of-the-art navigation system to learn a variety of parameters that control the robot's behaviour. The results show two easy to understand learned emotional states; the first is considered to be a model `fear', which increases obstacle avoidance while lowering speed when pain is induced or novelty is high. The second emotion is considered to be `happiness', which increases speed and lowers wall avoidance when pain is not present. Compared to the default non-adapting navigation system, the emotional responses decreased the overall number of collisions and improved time to navigate.

In this paper, we successfully apply GEFeS (Genetic & Evolutionary Feature Selection) to identify the key features in the human vaginal microbiome and in patient meta-data that are associated with bacterial vaginosis (BV). The vaginal microbiome is the community of bacteria found in a patient, and meta-data include behavioral practices and demographic information. Bacterial vaginosis is a disease that afflicts nearly one third of all women, but the current diagnostics are crude at best. We describe two types of classifies for BV diagnosis, and show that each is associated with one of two treatments. Our results show that the classifiers associated with the 'Treat Any Symptom' version have better performances that the classifier associated with the 'Treat Based on N-Score Value'. Our long term objective is to develop a more accurate and objective diagnosis and treatment of BV.

A cognitive mobile robot must be able to autonomously solve the three complex problems of navigating: where it is, where it is going and how it is going to get there. The first is addressed by techniques for simultaneous localization and mapping (SLAM). The next stage of navigating is to plan a path to a goal, which is often achieved by learning techniques due to the scale of search required. Commonly, the localisation and mapping stage is separated from path planning stage, with the function not of interest being considered ideal in order to simplify the problem (similarly, the goal is often predetermined by an external agent, such as a human operator specifying a location to reach). This work integrates the planning with the localisation and mapping in order to investigate the benefits of considering these aspects together (rather than as a separate functions as is often assumed). Firstly, experiments on real-robots show decreased localisation error in this approach (1.8 mm ±0.41 mm to 1.2 mm ±0.26 mm). Secondly, the number of steps to goal has concurrently been reduced (13.4 steps to 11.8 steps). This work is novel in the integration of evolutionary computation planning techniques with SLAM. It also has enabled the opportunity for rule-sharing between heterogeneous robots and the inclusion of action policies in SLAM filter updates.