This paper introduces a novel approach to enhancing cross-view localization, focusing on the fine-grained, sequential localization of street-view images within a single known satellite image patch, a significant departure from traditional one-to-one image retrieval methods. By expanding to sequential image fine-grained localization, our model, equipped with a novel Temporal Attention Module (TAM), leverages contextual information to significantly improve sequential image localization accuracy. Our method shows substantial reductions in both mean and median localization errors on the Cross-View Image Sequence (CVIS) dataset, outperforming current state-of-the-art single-image localization techniques. Additionally, by adapting the KITTI-CVL dataset into sequential image sets, we not only offer a more realistic dataset for future research but also demonstrate our model’s robust generalization capabilities across varying times and areas, evidenced by a 75.3% reduction in mean distance error in cross-view sequential image localization.

Seagrass meadows serve as critical carbon sinks, but estimating the amount of carbon they store requires knowledge of the seagrass species present. Underwater and surface vehicles equipped with machine learning algorithms can help to accurately estimate the composition and extent of seagrass meadows at scale. However, previous approaches for seagrass detection and classification have required supervision from patch-level labels. In this paper, we reframe seagrass classification as a weakly supervised coarse segmentation problem where image-level labels are used during training (25 times fewer labels compared to patch-level labeling) and patch-level outputs are obtained at inference time. To this end, we introduce SeaFeats, an architecture that uses unsupervised contrastive pre-training and feature similarity, and SeaCLIP, a model that showcases the effectiveness of large language models as a supervisory signal in domain-specific applications. We demonstrate that an ensemble of SeaFeats and SeaCLIP leads to highly robust performance. Our method outperforms previous approaches that require patch-level labels on the multi-species ‘DeepSeagrass’ dataset by 6.8% (absolute) for the class-weighted F1 score, and by 12.1% (absolute) for the seagrass presence/absence F1 score on the ‘Global Wetlands’ dataset. We also present two case studies for real-world deployment: outlier detection on the Global Wetlands dataset, and application of our method on imagery collected by the FloatyBoat autonomous surface vehicle.

Cross-view image geo-localization aims to determine the locations of outdoor robots by mapping current street-view images with GPS-tagged satellite image patches. Recent works have attained a remarkable level of accuracy in identifying which satellite patches the robot is in, where the location of the central pixel within the matched satellite patch is used as the robot coarse location estimation. This work focuses on robot fine-grained localization within a known satellite patch. Existing fine-grain localization work utilizes correlation operation to obtain similarity between satellite image local descriptors and street-view global descriptors. The correlation operation based on liner matching simplifies the interaction process between two views, leading to a large distance error and affecting model generalization. To address this issue, we devise a cross-view feature fusion network with self-attention and cross-attention layers to replace correlation operation. Additionally, we combine classification and regression prediction to further decrease location distance error. Experiments show that our novel network architecture outperforms the state-of-the-art, exhibiting better generalization capabilities in unseen areas. Specifically, our method reduces the median localization distance error by 43% and 50% respectively in the same area and unseen areas on the VIGOR benchmark.

Jamming Grippers are a novel class of soft robotic actuators that can robustly grasp and manipulate objects of arbitrary shape. They are formed by placing a granular material within a flexible skin connected to a vacuum pump and function by pressing the un-jammed gripper against a target object and evacuating the air to transition the material to a jammed (solid) state, gripping the target object. However, due to the complex interactions between grain morphology and target object shape, much uncertainty still remains regarding optimal constituent grain shapes for specific gripping applications. We address this challenge by utilising a modern Evolutionary Algorithm, NSGA-III, combined with a Discrete Element Method soft robot model to perform a multi-objective optimisation of grain morphology for use in jamming grippers for a range of target object sizes. Our approach optimises the microscopic properties of the system to elicit bespoke functional granular material performance driven by the complex relationship between the individual particle morphologies and the related emergent behaviour of the bulk state. Results establish the important contribution of grain morphology to gripper performance and the critical role of local surface curvature and the length scale governed by the relative sizes of the grains and target object.

Learning embeddings that are invariant to the pose of the object is crucial in visual image retrieval and re-identification. The existing approaches for person, vehicle, or animal re-identification tasks suffer from high intra-class variance due to deformable shapes and different camera viewpoints. To overcome this limitation, we propose to align the image embedding with a predefined order of the key-points. The proposed keypoint aligned embeddings model (KAE-Net) learns part-level features via multi-task learning which is guided by keypoint locations. More specifically, KAE-Net extracts channels from a feature map activated by a specific keypoint through learning the auxiliary task of heatmap reconstruction for this keypoint. The KAE-Net is compact, generic and conceptually simple. It achieves state of the art performance on the benchmark datasets of CUB-200-2011, Cars196 and VeRi-776 for retrieval and re-identification tasks.

Inferring 3D scene information from 2D observations is an open problem in computer vision. We propose using a deep-learning based energy minimization framework to learn a consistency measure between 2D observations and a proposed world model, and demonstrate that this framework can be trained end-to-end to produce consistent and realistic inferences. We evaluate the framework on human pose estimation and voxel-based object reconstruction benchmarks and show competitive results can be achieved with relatively shallow networks with drastically fewer learned parameters and floating point operations than conventional deep-learning approaches.

This research considers the task of evolving the physical structure of a robot to enhance its performance in various environments, which is a significant problem in the field of Evolutionary Robotics. Inspired by the fields of evolutionary art and sculpture, we evolve only targeted parts of a robot, which simplifies the optimisation problem compared to traditional approaches that must simultaneously evolve both (actuated) body and brain. Exploration fidelity is emphasised in areas of the robot most likely to benefit from shape optimisation, whilst exploiting existing robot structure and control. Our approach uses a Genetic Algorithm to optimise collections of Bezier splines that together define the shape of a legged robot's tibia, and leg performance is evaluated in parallel in a high-fidelity simulator. The leg is represented in the simulator as 3D-printable file, and as such can be readily instantiated in reality. Provisional experiments in three distinct environments show the evolution of environment-specific leg structures that are both high-performing and notably different to those evolved in the other environments. This proof-of-concept represents an important step towards the environment-dependent optimisation of performance-critical components for a range of ubiquitous, standard, and already-capable robots that can carry out a wide variety of tasks.

We propose Adversarially Parameterized Optimization, a framework for learning low-dimensional feasible parameterizations of human poses and inferring 3D poses from 2D input. We train a Generative Adversarial Network to `imagine' feasible poses, and search this imagination space for a solution that is consistent with observations. The framework requires no scene/observation correspondences and enforces known geometric invariances without dataset augmentation. The algorithm can be configured at run time to take advantage of known values such as intrinsic/extrinsic camera parameters or target height when available without additional training. We demonstrate the framework by inferring 3D human poses from projected joint positions for both single frames and sequences. We show competitive results with extremely simple shallow network architectures and make the code publicly available.

Driver training is one of the interventions aimed at mitigating the number of crashes that involve novice drivers. Our failure to understand what is really important for learners, in terms of risky driving, is one of the many drawbacks restraining us from building better training programs. Currently, there is a need to develop and evaluate advanced driving assistance systems that could comprehensively assess driving competencies. The aim of this paper is to present a novel intelligent driver training system that analyzes crash risks for a given driving situation, providing avenues for the improvement and personalization of driver training programs. The analysis takes into account numerous variables synchronously acquired from the driver, the vehicle, and the environment. The system then segments out the maneuvers within a drive. This paper further presents the fuzzy set theory to develop the safety inference rules for each maneuver executed during the drive, and presents a framework and its associated prototype that can be used to comprehensively view and assess complex driving maneuvers and then provides a comprehensive analysis of the drive used to give feedback to novice drivers.

Monitoring and estimation of marine populations is of paramount importance for the conservation and management of sea species. Regular surveys are used to this purpose followed often by a manual counting process. This paper proposes an algorithm for automatic detection of dugongs from imagery taken in aerial surveys. Our algorithm exploits the fact that dugongs are rare in most images, therefore we determine regions of interest partially based on color rarity. This simple observation makes the system robust to changes in illumination. We also show that by applying the extended-maxima transform on red-ratio images, submerged dugongs with very fuzzy edges can be detected. Performance figures obtained here are promising in terms of degree of confidence in the detection of marine species, but more importantly our approach represents a significant step in automating this type of surveys.

Maintenance trains travel in convoy. In Australia, only the first train of the convoy pays attention to the track signalization (the other convoy vehicles simply follow the preceding vehicle). Because of human errors, collisions can happen between the maintenance vehicles. Although an anti-collision system based on a laser distance meter is already in operation, the existing system has a limited range due to the curvature of the tracks. In this paper, we introduce an anti-collision system based on vision. The two main ideas are, (1) to warp the camera image into an image where the rails are parallel through a projective transform, and (2) to track the two rail curves simultaneously by evaluating small parallel segments. The performance of the system is demonstrated on an image dataset.

It is easy to create new combinatorial games but more difficult to predict those that will interest human players. We examine the concept of game quality, its automated measurement through self-play simulations, and its use in the evolutionary search for new high-quality games. A general game system called Ludi is described and experiments conducted to test its ability to synthesize and evaluate new games. Results demonstrate the validity of the approach through the automated creation of novel, interesting, and publishable games.

Perceptual aliasing makes topological navigation a difficult task. In this paper we present a general approach for topological SLAM (simultaneous localisation and mapping) which does not require motion or odometry information but only a sequence of noisy measurements from visited places. We propose a particle filtering technique for topological SLAM which relies on a method for disambiguating places which appear indistinguishable using neighbourhood information extracted from the sequence of observations. The algorithm aims to induce a small topological map which is consistent with the observations and simultaneously estimate the location of the robot. The proposed approach is evaluated using a data set of sonar measurements from an indoor environment which contains several similar places. It is demonstrated that our approach is capable of dealing with severe ambiguities and, and that it infers a small map in terms of vertices which is consistent with the sequence of observations.

In this paper we present a system for appearance-based topological mapping and localisation using vision data. The algorithms are designed for robots which are equipped with FPGA cameras. Such cameras do not provide the entire image to the robot but simple image features like colour histograms.

In this paper we address the problem of topologically mapping environments which contain inherent perceptual aliasing caused by repeated environment structures. We propose an approach that does not use motion or odometric information but only a sequence of deterministic measurements observed by traversing an environment. Our algorithm implements a stochastic local search to build a small map which is consistent with local adjacency information extracted from a sequence of observations. Moreover, local adjacency information is incorporated to disambiguate places which are physically different but appear identical to the robots senses. Experiments show that the proposed method is capable of mapping environments with a high degree of perceptual aliasing, and that it infers a small map quickly.

Maintenance trains travel in convoy. In Australia, only the first train of the convoy pays attention to the track signalization (the other convoy vehicles simply follow the preceding vehicle). Because of human errors, collisions can happen between the maintenance vehicles. Although an anti-collision system based on a laser distance meter is already in operation, the existing system has a limited range due to the curvature of the tracks. In this paper, we introduce an anti-collision system based on vision. The proposed system induces a 3D model of the track as a piecewise quadratic function (with continuity constraints on the function and its derivative). The geometric constraints of the rail tracks allow the creation of a completely self-calibrating system. Although road lane marking detection algorithms perform well most of the time for rail detection, the metallic surface of a rail does not always behave like a road lane marking. Therefore we had to develop new techniques to address the specific problems of the reflectance of rails.

In this paper we introduce methods to build a SOM that can be used as an isometric map for mobile robots. That is, given a dataset of sensor readings collected at points uniformly distributed with respect to the ground, we wish to build a SOM whose neurons (prototype vectors in sensor space) correspond to points uniformly distributed on the ground. Manifold learning techniques have already been used for dimensionality reduction of sensor space in navigation systems. Our focus is on the isometric property of the SOM. For reliable path-planning and information sharing between several robots, it is desirable that the robots build an internal representation of the sensor manifold, a map, that is isometric with the environment. We show experimentally that standard Non-Linear Dimensionality Reduction (NLDR) algorithms do not provide isometric maps for range data and bearing data. However, the auxiliary low dimensional manifolds created can be used to improve the distribution of the neurons of a SOM (that is, make the neurons more evenly distributed with respect to the ground). We also describe a method to create an isometric map from a sensor readings collected along a polygonal line random walk.

The support vector machine (SVM) solution corresponds to the centre of the largest sphere inscribed in version space. Alternative approaches like Bayesian point machines (BPM) and analytic centre machines have suggested that the generalization performance can be further enhanced by considering other possible centres of version space like the centroid (centre of mass) or the analytic centre. We present an algorithm to compute exactly the centroid of higher dimensional polyhedra, then derive approximation algorithms to build a new learning machine whose performance is comparable to BPM. We also show that for regular kernel matrices (Gaussian kernels for example), the SVM solution can be obtained by solving a linear system of equalities.

Validity interval analysis (VIA) is a generic tool for analyzing the input-output behavior of feedforward neural networks. VIA is a rule extraction technique that relies on a rule refinement algorithm. The rules are of the form R(i)-->R(0) which reads if the input of the neural network is in the region R(i), then its output is in the region R(0), where regions are axis parallel hypercubes. VIA conjectures, then refines and checks rules for inconsistency. This process can be computationally expensive, and the rule refinement phase becomes critical. Hence, the importance of knowing the complexity of these rule refinement algorithms. In this paper, we show that the rule refinement part of VIA always converges in one run for single-weight-layer networks, and has an exponential average rate of convergence for multilayer networks. We also discuss some variations of the standard VIA formulae.

This study empirically investigates variations of hill climbing algorithms for training artificial neural networks on the 5-bit parity classification task. The experiments compare the algorithms when they use different combinations of random number distributions, variations in the step size and changes of the neural networks' initial weight distribution. A hill climbing algorithm which uses inline search is proposed. In most experiments on the 5-bit parity task it performed better than simulated annealing and standard hill climbing.

We present a method to unify the rules obtained by the M-of-N rule-extraction technique. The rules extracted from a perceptron by the M-of-N algorithm are in correspondence with sets of minimal Boolean vectors with respect to the classical partial order defined on vectors. Our method relies on a simple characterization of another partial order defined on Boolean vectors. We show that there exists also a correspondence between sets of minimal Boolean vectors with respect to this order and M-of-N rules equivalent to a perceptron. The gain is that fewer rules are generated with the second order. Independently, we prove that deciding whether a perceptron is symmetric with respect to two variables is NP-complete.