Real-time 3D Gaussian Splatting (3DGS) has emerged as an efficient, high-fidelity alternative to neural radiance fields for novel view synthesis, enabling second-scale training and rendering via GPU rasterization. However, when input image collections contain transient disturbances (e.g., dynamic objects, exposure variations, motion blur) or suffer from sparse view coverage at scene boundaries, 3DGS performance degrades significantly due to reconstruction artifacts such as ghosting, floating points, and blurred surfaces. In this work, we present a unified framework that jointly addresses two types of artifacts: (1) corruption-driven artifacts, caused by transient or occluded content; and (2) information-limited artifacts, caused by insufficient multi-view observations. Our method leverages the training gradient signal, as well as the shape and spatial distribution of Gaussians, to adaptively suppress unreliable splats through a soft-masking strategy, without relying on any pretrained segmentation or feature networks. Extensive experiments on two real-world datasets with dynamic scenes and sparse camera trajectories demonstrate that our approach outperforms state-of-the-art robust 3DGS and uncertainty-pruning techniques in artifact suppression and reconstruction fidelity, while preserving real-time performance.

Test-Time Adaptation (TTA) enables deep neural networks to handle distribution shifts without requiring labels at inference. However, existing methods commonly assume complete class overlap between source and target domains, which rarely holds in practice. We study the challenging setting of Partial Class Imbalance, where the target domain contains only a subset of source classes. We show that entropy minimization–based TTA methods degrade over long test sequences because batch normalization updates bias feature representations toward visible classes, resulting in skewed predictions. To address this, we propose Logit-Adjusted Entropy Minimization, a simple yet effective strategy that integrates target class priors into the adaptation objective. Our method is model-agnostic and can be seamlessly applied to a wide range of TTA algorithms. Extensive experiments on CIFAR-100-C, ImageNetC under diverse corruptions and severity levels, and the large-scale DomainNet-126 dataset demonstrate that our method consistently improves adaptation stability and accuracy for both CNNs and Vision Transformers. Compared to strong baselines, our approach reduces overfitting to visible classes and mitigates performance degradation in long-sequence adaptation. Code is available at https://github.com/thilinawee/latta

The paper presents an innovative non-destructive inspection (NDI) method, via employing a custom vision-based machine learning model, for evaluating the quality of Self-Piercing Rivet (SPR) joints, a commonly used joining technique in light vehicle production. Non-destructive inspection (NDI) is crucial for structural integrity analyses, including fracture and fatigue assessment. Utilizing μ-CT (Micro-Computed Tomography) scans of SPR joints and the Machine Vision AI (artificial intelligence) model, the present approach enables the detection and quantification of key quality parameters. The AI model is trained on images extracted from a comprehensive dataset of μ-CT scans of modified SPR joints. Samples used for training are modified carefully to represent the combined defects commonly found in SPR joints. The trained vision-based AI model is deployed to identify and quantify defects, accounting for cumulative errors from material inconsistencies, manufacturing imperfections, and dimensional tolerances. These parameters comprise cumulative defects that may affect joint integrity.The proposed method provides valuable cumulative measured inputs to numerical models of whole riveted structure, non-destructively, which are essential for evaluating the performance of complex structures incorporating SPRs. By significantly enhancing the prediction of the effects of manufacturing defects on lightweight structures, including fatigue life and joint durability, this approach enables more accurate and reliable assessments of individual joints; unlike current techniques, which depend on assumed, averaged, or estimated defect parameters without direct inspection. Ultimately, the AI-based NDI tool enables the enhancement of error detection precision, contributing to safer and more reliable structural integrity evaluations for a variety of manufacturing applications.

To be adopted in safety-critical domains like medical image analysis, AI systems must provide human-interpretable decisions. Variational Information Pursuit (VIP) offers an interpretable-by-design framework by sequentially querying input images for human-understandable concepts, using their presence or absence to make predictions. However, existing V-IP methods overlook sample-specific uncertainty in concept predictions, which can arise from ambiguous features or model limitations, leading to suboptimal query selection and reduced robustness. In this paper, we propose an interpretable and uncertainty-aware framework for medical imaging that addresses these limitations by accounting for upstream uncertainties in concept-based, interpretable-by-design models. Specifically, we introduce two uncertainty-aware models, EUAV-IP and IUA-VIP, that integrate uncertainty estimates into the V-IP querying process to prioritize more reliable concepts per sample. EUAV-IP skips uncertain concepts via masking, while IUAV-IP incorporates uncertainty into query selection implicitly for more informed and clinically aligned decisions. Our approach allows models to make reliable decisions based on a subset of concepts tailored to each individual sample, without human intervention, while maintaining overall interpretability. We evaluate our methods on five medical imaging datasets across four modalities: dermoscopy, X-ray, ultrasound, and blood cell imaging. The proposed IUAV-IP model achieves state-of-the-art accuracy among interpretable-by-design approaches on four of the five datasets, and generates more concise explanations by selecting fewer yet more informative concepts. These advances enable more reliable and clinically meaningful outcomes, enhancing model trustworthiness and supporting safer AI deployment in healthcare. Our code and models are available at: https://github.com/Nahiduzzaman09/ UAV-IP.

This paper addresses a gap between the capabilities and utilisation of robotics and automation in laboratory settings and builds upon the concept of Self Driving Labs (SDL). We introduce an innovative approach to the temporal characterisation of materials. The article discusses the challenges posed by manual methods involving established laboratory equipment and presents an automated hyperspectral characterisation station. This station integrates robot-aided hyperspectral imaging (HSI), complex material characterisation modelling, and automated data analysis, offering a non-destructive and comprehensive approach. This work explains how the proposed assembly can automatically measure the half-life of biodegradable polymers with higher throughput and accuracy than manual methods. The investigation explores the effect of pH, number of average molecular weight (Mn), end groups, and blends on the degradation rate of polylactic acid (PLA). The novel contributions of the paper lie in introducing an adaptable classification station for characterisation and presenting an innovative methodology for polymer degradation rate measurements. The proposed system holds promise for expediting the development of high-throughput screening and characterisation methods within advanced material and chemistry laboratories. Note to Practitioners—The characterisation and classification of materials hold significant importance within the realms of materials science, chemistry, manufacturing and the circular economy. We introduce an innovative approach by employing robotics to automate hyperspectral imaging and processing to accelerate and improve material characterisation within a laboratory environment. This methodology facilitates the time-dependent characterisation of materials. This automated system encompasses sample manipulation and handling for hyperspectral scanning, accompanied by automated image and data processing procedures to minimise manual interventions effectively. The developed system can be seamlessly integrated into lab settings, offering objective, accurate, and automated measurement of biodegradable polymer degradation rates.

Domain adaptation is a well-studied field in machine learning. Distribution shift between train (source) and test (target) datasets is a common problem encountered in machine learning applications. One approach to resolve this issue is to use the Unsupervised Domain Adaptation (UDA) technique that carries out knowledge transfer from a label-rich source domain to an unlabeled target domain. Outliers that exist in either source or target datasets can introduce additional challenges when using UDA in practice. In this article, \(\alpha\) -divergence is used as a measure to minimize the discrepancy between the source and target distributions while inheriting robustness, adjustable with a single parameter \(\alpha\) , as the prominent feature of this measure. Here, it is shown that the other well-known divergence-based UDA techniques can be derived as special cases of the proposed method. Furthermore, a theoretical upper bound is derived for the loss in the target domain in terms of the source loss and the \(\alpha\) -divergence between the joint distributions in the two domains. The robustness of the proposed method is validated through testing on several benchmarked datasets in open-set and partial UDA setups where extra classes existing in target and source datasets are considered as outliers. The code is publicly available at https://github.com/rashidis/IT-RUDA .

Advanced machine vision and deep learning models are now widely used as virtual sensors in various applications to detect and classify objects in image measurements. Typically, these virtual sensors output measurements as a set of class probabilities for each detected object within the sensor’s field of view. However, integrating this type of data into multi-target tracking systems, traditionally based on point measurement detections, presents some challenges. This paper proposes a solution by introducing a labeled multi-Bernoulli filter formulated to handle detections with class measurements from virtual sensors for multi-target tracking. Furthermore, we explore the application of this filter for multi-sensor multi-target tracking within a distributed sensor network. Through numerical experiments involving vehicles equipped with cameras and deep learning classification modules navigating complex roads, we demonstrate that incorporating class information into the tracking process improves tracking accuracy. This improvement is further enhanced when vehicles share and fuse their information in a distributed manner. Our findings highlight the benefits of integrating class probability data into multi-target tracking filters, offering substantial improvements in the accuracy of automated monitoring systems in dynamic and complex environments.

Deep learning techniques often perform poorly in the presence of domain shift, where the test data follows a different distribution than the training data. The most practically desirable approach to address this issue is Single Domain Generalization (S-DG), which aims to train robust models using data from a single source. Prior work on S-DG has primarily focused on using data augmentation techniques to generate diverse training data. In this paper, we explore an alternative approach by investigating the robustness of linear operators, such as convolution and dense layers commonly used in deep learning. We propose a novel operator called "XCNorm" that computes the normalized cross-correlation between weights and an input feature patch. This approach is invariant to both affine shifts and changes in energy within a local feature patch and eliminates the need for commonly used non-linear activation functions. We show that deep neural networks composed of this operator are robust to common semantic distribution shifts. Furthermore, our empirical results on single-domain generalization benchmarks demonstrate that our proposed technique performs comparably to the stateof-the-art methods.1

In this paper, we propose a lightweight approach for industrial defect detection that is modeled based on anomaly detection using point pattern data. Most recent works use global features for feature extraction to summarize image content. However, global features are not robust against lighting and viewpoint changes and do not describe images’ geometrical information which, thus, cannot be fully utilized in manufacturing industry. To the best of our knowledge, we are the first to propose using transfer learning of local/point pattern features to overcome these limitations and capture geometrical information of the interested image regions. We model these local/point pattern features as a random finite set (RFS). In addition, we propose RFS energy, in contrast to RFS likelihood as an anomaly score. The similarity distribution of point pattern features of the normal sample has been modeled as a multivariate Gaussian. Parameters learning of the proposed RFS energy does not require any heavy computation. We evaluate the proposed approach on the MVTec AD dataset, a multi-object defect detection dataset. Experimental results show the outstanding performance of our proposed approach compared to the state-of-the-art methods, and the proposed RFS energy outperforms the state-of-the-art in the few shot learning settings. Also, we evaluated the proposed approach on needle dataset, a real life application, to detect defective needle samples given three different views. The results show superior performance compared to DiffeNet.

This paper introduces an approach to distributed multi-object tracking for connected vehicles, aiming to overcome the inherent challenges of label inconsistency and double counting prevalent in distributed information fusion methods, particularly in the context of situational awareness for connected vehicles. Our proposed method expands the label space by incorporating sensor identity into the object’s label. Furthermore, we present an intuitive merging algorithm designed to effectively eliminate instances of double counting. The approach is formulated, and an algorithm is developed for implementation within a labeled multi-Bernoulli filter, executed locally on each node of a distributed network responsible for information fusion. To assess the efficacy of our solution, we evaluate its performance in a highly demanding scenario specifically designed for intelligent transport systems and compare its performance against alternative approaches.

Despite significant amount of work reported in the computer vision literature, segmenting images or videos based on multiple cues such as objectness, texture and motion, is still a challenge. This is particularly true when the number of objects to be segmented is not known or there are objects that are not classified in the training data (unknown objects). A possible remedy to this problem is to utiize graph-based clustering techniques such as Correlation Clustering. It is known that using long range affinities (Lifted multicut), makes correlation clustering more accurate than using only adjacent affinities (Multicut). However, the former is computationally expensive and hard to use. In this paper, we introduce a new framework to perform image/motion segmentation using an affinity learning module and a Message Passing Graph Neural Network (MPGNN). The affinity learning module uses a permutation invariant affinity representation to overcome the multi-object problem. The paper shows, both theoretically and empirically, that the proposed MPGNN aggregates higher order information and thereby converts the Lifted Multicut Problem (LMP) to a Multicut Problem (MP), which is easier and faster to solve. Importantly, the proposed method can be generalized to deal with different clustering problems with the same MPGNN architecture. For instance, our method produces competitive results for single image segmentation (on BSDS dataset) as well as unsupervised video object segmentation (on DAVIS17 dataset), by only changing the feature extraction part. In addition, using an ablation study on the proposed MPGNN architecture, we show that the way we update the parameterized affinities directly contributes to the accuracy of the results.

Deep convolutional neural networks for dense prediction tasks are commonly optimized using synthetic data, as generating pixel-wise annotations for real-world data is laborious. However, the synthetically trained models do not generalize well to real-world environments. This poor "synthetic to real" (S2R) generalization we address through the lens of shortcut learning. We demonstrate that the learning of feature representations in deep convolutional networks is heavily influenced by synthetic data artifacts (shortcut attributes). To mitigate this issue, we propose an Information-Theoretic Shortcut Avoidance (ITSA) approach to automatically restrict shortcut-related information from being encoded into the feature representations. Specifically, our proposed method minimizes the sensitivity of latent features to input variations: to regularize the learning of robust and shortcut-invariant features in synthetically trained models. To avoid the prohibitive computational cost of direct input sensitivity optimization, we propose a practical yet feasible algorithm to achieve robustness. Our results show that the proposed method can effectively improve S2R generalization in multiple distinct dense prediction tasks, such as stereo matching, optical flow, and semantic segmentation. Importantly, the proposed method enhances the robustness of the synthetically trained networks and outperforms their fine-tuned counterparts (on real data) for challenging out-of-domain applications.

Tracking multiple objects through time is an important part of an intelligent transportation system. Random finite set (RFS)-based filters are one of the emerging techniques for tracking multiple objects. In multi-object tracking (MOT), a common assumption is that each object is moving independent of its surroundings. However, in many real-world applications, objects interact with one another and the environment. Such interactions are rarely considered within the tracking process. In this paper, we present a novel approach to incorporate target interactions within the prediction step of a RFS-based multi-target filter, i.e. labeled multi-Bernoulli (LMB) filter. The proposed method is capable of explicitly incorporating the effect of target interactions within the filtering process, with little to no changes made for specific applications. We have developed the proposed filter for two practical applications, i.e. tracking a coordinated swarm and vehicles. The method has been tested for a complex vehicle tracking dataset and compared with the PHD filter and LMB filter through the OSPA metric, and with the LMB filter through OSPA(2) metric and cardinality error. The results demonstrate that the proposed interaction-aware method depicts considerable performance enhancement over the other methods in terms of the selected metrics.

State-of-the-art stereo matching networks trained only on synthetic data often fail to generalize to more challenging real data domains. In this paper, we attempt to unfold an important factor that hinders the networks from generalizing across domains: through the lens of shortcut learning. We demonstrate that the learning of feature representations in stereo matching networks is heavily influenced by synthetic data artefacts (shortcut attributes). To mitigate this issue, we propose an Information-Theoretic Shortcut Avoidance (ITSA) approach to automatically restrict shortcut-related information from being encoded into the feature representations. As a result, our proposed method learns robust and shortcut-invariant features by minimizing the sensitivity of latent features to input variations. To avoid the prohibitive computational cost of direct input sensitivity optimization, we propose an effective yet feasible algorithm to achieve robustness. We show that using this method, state-of-the-art stereo matching networks that are trained purely on synthetic data can effectively generalize to challenging and previously unseen real data scenarios. Importantly, the proposed method enhances the robustness of the synthetic trained networks to the point that they outperform their fine-tuned counterparts (on real data) for challenging out-of-domain stereo datasets.

Maximisation of Consensus (MaxCon) is one of the most widely used robust criteria in computer vision. Tennakoon et al. (CVPR2021), made a connection between MaxCon and estimation of influences of a Monotone Boolean function. In such, there are two distributions involved: the distribution defining the influence measure; and the distribution used for sampling to estimate the influence measure. This paper studies the concept of weighted influences for solving MaxCon. In particular, we study the Bernoulli measures. Theoretically, we prove the weighted influences, under this measure, of points belonging to larger structures are smaller than those of points belonging to smaller structures in general. We also consider another “natural” family of weighting strategies: sampling with uniform measure concentrated on a particular (Hamming) level of the cube. One can choose to have matching distributions: the same for defining the measure as for implementing the sampling. This has the advantage that the sampler is an unbiased estimator of the measure. Based on weighted sampling, we modify the algorithm of Tennakoon et al., and test on both synthetic and real datasets. We show some modest gains of Bernoulli sampling, and we illuminate some of the interactions between structure in data and weighted measures and weighted sampling.

In autonomous vehicles, depth information for the environment surrounding the vehicle is commonly extracted using time-of-flight (ToF) sensors such as LiDARs and RADARs. Those sensors have some limitations that may potentially degrade the quality and utility of the depth information to a substantial extent. An alternative solution is depth estimation from stereo pairs. However, stereo matching and depth estimation often fails at ill-posed regions including areas with repetitive patterns or textureless surfaces which are commonly found on planar surfaces. This paper focuses on designing an efficient framework for stereo depth estimation, using deep learning technique, that is robust against the mentioned ill-posed regions. With the observation that disparities of all pixels belonging to planar areas (scene plane) viewed by two rectified stereo images can be described using affine transformations, our proposed method predicts pixel-wise affine transformation parameters based on the depth information encoded in the aggregated cost volume. We also introduce a propagation term which enforces all pixels belonging to the same scene plane to be transformed using the same parameters. Disparity can then be computed by multiplying the predicted affine parameters with the corresponding pixel locations. The proposed method was evaluated on several benchmark datasets. We are able to obtain competitive results and at the same time reducing the processing time of common convolution neural network (CNN) in stereo matching by 50%. Analysis of the findings shows that our method can produce reliable results at the ill-posed regions which are challenging to the current state-of-the-arts methods.

Autonomous vehicles in intelligent transportation systems must be able to perform reliable and safe navigation. This necessitates accurate object detection, which is commonly achieved by high-precision depth perception. Existing stereo vision-based depth estimation systems generally involve computation of pixel correspondences and estimation of disparities between rectified image pairs. The estimated disparity values will be converted into depth values in downstream applications. As most applications often work in the depth domain, the accuracy of depth estimation is often more compelling than disparity estimation. However, at large distances (> 50m), the accuracy of disparity estimation does not directly translate to the accuracy of depth estimation. In the context of learning-based stereo systems, this is mainly due to biases imposed by the choices of the disparity-based loss function and the training data. Consequently, the learning algorithms often produce unreliable depth estimates of under-represented foreground objects, particularly at large distances. To resolve this issue, we first analyze the effect of those biases and then propose a pair of depth-based loss functions for foreground objects and background separately. These loss functions can be tuned and can balance the inherent bias of the stereo learning algorithms. The efficacy of our solution is demonstrated by an extensive set of experiments, which are benchmarked against state of the art. We show on the KITTI 2015 benchmark that our proposed solution yields substantial improvements in disparity and depth estimation, particularly for objects located at distances beyond 50 meters, outperforming the previous state of the art by 10%.

Consensus maximisation (MaxCon), which is widely used for robust fitting in computer vision, aims to find the largest subset of data that fits the model within some tolerance level. In this paper, we outline the connection between MaxCon problem and the abstract problem of finding the maximum upper zero of a Monotone Boolean Function (MBF) defined over the Boolean Cube. Then, we link the concept of influences (in a MBF) to the concept of outlier (in MaxCon) and show that influences of points belonging to the largest structure in data would generally be smaller under certain conditions. Based on this observation, we present an iterative algorithm to perform consensus maximisation. Results for both synthetic and real visual data experiments show that the MBF based algorithm is capable of generating a near optimal solution relatively quickly. This is particularly important where there are large number of outliers (gross or pseudo) in the observed data.

This paper presents an innovative method for motion segmentation in RGB-D dynamic videos with multiple moving objects. The focus is on finding static, small or slow moving objects (often overlooked by other methods) that their inclusion can improve the motion segmentation results. In our approach, semantic object based segmentation and motion cues are combined to estimate the number of moving objects, their motion parameters and perform segmentation. Selective object-based sampling and correspondence matching are used to estimate object specific motion parameters. The main issue with such an approach is the over segmentation of moving parts due to the fact that different objects can have the same motion (e.g. background objects). To resolve this issue, we propose to identify objects with similar motions by characterizing each motion by a distribution of a simple metric and using a statistical inference theory to assess their similarities. To demonstrate the significance of the proposed statistical inference, we present an ablation study, with and without static objects inclusion, on SLAM accuracy using the TUM-RGBD dataset. To test the effectiveness of the proposed method for finding small or slow moving objects, we applied the method to RGB-D MultiBody and SBM-RGBD motion segmentation datasets. The results showed that we can improve the accuracy of motion segmentation for small objects while remaining competitive on overall measures.

Volumetric imaging is an essential diagnostic tool for medical practitioners. The use of popular techniques such as convolutional neural networks (CNN) for analysis of volumetric images is constrained by the availability of detailed (with local annotations) training data and GPU memory. In this paper, the volumetric image classification problem is posed as a multi-instance classification problem and a novel method is proposed to adaptively select positive instances from positive bags during the training phase. This method uses the extreme value theory to model the feature distribution of the images without a pathology and use it to identify positive instances of an imaged pathology. The experimental results, on three separate image classification tasks (i.e. classify retinal OCT images according to the presence or absence of fluid build-ups, emphysema detection in pulmonary 3D-CT images and detection of cancerous regions in 2D histopathology images) show that the proposed method produces classifiers that have similar performance to fully supervised methods and achieves the state of the art performance in all examined test cases.

Classification of 3D shapes into physically meaningful categories is one of the most important tasks in understanding the immediate environment. Methods that leverage the recent advancements in deep learning have shown to outperform the traditional approaches. However, performances of those methods have only been analyzed with relatively clean data. Three-dimensional measurement sets (point clouds) produced by 3D scanners are rarely that accurate and often contain noise, outliers or missing points. This paper presents an extensive analysis of the robustness of the state-of-the-art neural network algorithms to realistic data inaccuracies. Our experiments show that the existence of these inaccuracies can significantly affect the performance of the deep learning-based algorithms.

Retinal swelling due to the accumulation of fluid is associated with the most vision-threatening retinal diseases. Optical coherence tomography (OCT) is the current standard of care in assessing the presence and quantity of retinal fluid and image-guided treatment management. Deep learning methods have made their impact across medical imaging, and many retinal OCT analysis methods have been proposed. However, it is currently not clear how successful they are in interpreting the retinal fluid on OCT, which is due to the lack of standardized benchmarks. To address this, we organized a challenge RETOUCH in conjunction with MICCAI 2017, with eight teams participating. The challenge consisted of two tasks: fluid detection and fluid segmentation. It featured for the first time: all three retinal fluid types, with annotated images provided by two clinical centers, which were acquired with the three most common OCT device vendors from patients with two different retinal diseases. The analysis revealed that in the detection task, the performance on the automated fluid detection was within the inter-grader variability. However, in the segmentation task, fusing the automated methods produced segmentations that were superior to all individual methods, indicating the need for further improvements in the segmentation performance.

In this paper we present a novel non-Bayesian filtering method for tracking multiple objects with a particular application in time-lapse cell microscopic video sequence. In our method the heat-map of the frame sequence is extracted and represented as a pseudo-probability hypothesis density of the image. The pseudo-probability hypothesis density is used as measurements and fused with a prior Poisson random finite set density. We employed Cauchy-Schwarz divergence for information fusion. The presented algorithm was tested on a publicly available cell microscopic video sequence.

Cell life-cycle and motility analysis is one of the fundamental tasks in many biological research activities, and its automation is a challenging problem. To solve the cell tracking problem using a Bayesian stochastic filter, one needs to properly incorporate all the information available about the cell behavior within the filter. This includes its movements, changes during mitosis process (up to splitting) and death. This paper demonstrates an effective way to perform this task for a particular family of cells (Chinese Hamster Ovarian (CHO) cells) that are known to have a commonly elliptic shape, and elongate during their mitosis. We model this by incorporating an ellipse-based state variable, a particular spawning process and an intuitive adaptive (measurement-driven) birth process that are added to the prediction step of a multi-Bernoulli filter. Our numerical experimental results involving microscopic images of living cells demonstrate significant improvement in tracking performance as a result of the proposed additions.

With the recent advent of labeled random finite set filters, it is now possible to not only estimate the number of objects and their states, but also track their trajectories all within the stochastic filtering scheme. This paper investigates how the objects label information returned by a labeled multi-Bernoulli filter can be effectively used for sensor control purposes. The main focus is on selective multi-object tracking applications where objects with particular labels are of high priority, and the sensor needs to be controlled to achieve maximum confidence in tracking performance of the filter for those objects. We formulated and examined two novel solutions. The first is based on maximization of the confidence in what the filter returns in relation to the existence of the objects of interest. The second solution is based on maximization of the confidence in what the filter returns in relation to the both existence and the states of the objects of interest. We also present an intuitive solution for tracking scenarios when some of the targets of interest temporarily disappear then reappear. Simulation results indicate how the proposed methods can lead to significant improvements in terms of tracking accuracy of objects of interest, compared to using the generic (non-selective) sensor control methods.

Condition monitoring of storm-water pipe systems are carried-out regularly using semi-automated processors. Semi-automated inspection is time consuming, expensive and produces varying and relatively unreliable results due to operators fatigue and novicity. This paper propose an innovative method to automate the storm-water pipe inspection and condition assessment process which employs a computer vision algorithm based on deep-neural network architecture to classify the defect types automatically. With the proposed method, the operator only needs to guide the robot through each pipe and no longer needs to be an expert. The results obtained on a CCTV video dataset of storm-water pipes shows that the deep neural network architectures trained with data augmentation and transfer learning is capable of achieving high accuracies in identifying the defect types.

This paper approaches the problem of geometric multi-model fitting as a data segmentation problem. The proposed solution is based on a sequence of sampling hyperedges from a hypergraph, model selection and hypergraph clustering steps. We developed a sampling method that significantly facilitates solving the segmentation problem using a new form of the Markov-Chain-Monte-Carlo (MCMC) method to effectively sample from hyperedge distribution. To sample from this distribution effectively, our proposed Markov Chain includes new ways of long and short jumps to perform exploration and exploitation of all structures. To enhance the quality of samples, a greedy algorithm is used to exploit nearby structure based on the minimization of the Least k th Order Statistics cost function . Unlike common sampling methods, ours does not require any specific prior knowledge about the distribution of models. The output set of samples leads to a clustering solution by which the final model parameters for each segment are obtained. The method competes favorably with the state-of-the-art both in terms of computation power and segmentation accuracy.

Identifying the underlying models in a set of data points that is contaminated by noise and outliers leads to a highly complex multi-model fitting problem. This problem can be posed as a clustering problem by the projection of higher-order affinities between data points into a graph, which can be clustered using spectral clustering. Calculating all possible higher-order affinities is computationally expensive. Hence, in most cases, only a subset is used. In this paper, we propose an effective sampling method for obtaining a highly accurate approximation of the full graph, which is required to solve multi-structural model fitting problems in computer vision. The proposed method is based on the observation that the usefulness of a graph for segmentation improves as the distribution of the hypotheses that are used to build the graph approaches the distribution of the actual parameters for the given data. In this paper, we approximate this actual parameter distribution by using a th-order statistics-based cost function, and the samples are generated using a greedy algorithm that is coupled with a data sub-sampling strategy. The experimental analysis shows that the proposed method is both accurate and computationally efficient compared with the state-of-the-art robust multi-model fitting techniques. The implementation of the method is publicly available from https://github.com/RuwanT/model-fitting-cbs.

We present a new image feature detection method. Our method selects features based on segmenting points with high local intensity variations across different scales using a robust rank order statistics approach. Our method produces a large number of repeatable features that are invariant to several image transformations such as rotation, scaling, viewpoint, and lighting variations. We show the advantages of our feature in comparison to other existing features using the Oxford dataset. We also show that, when used in monocular and stereo SLAM systems, our feature outperforms SIFT in terms of the pose estimation accuracy using several public datasets including the KITTI dataset.

A constrained sensor control method is presented for multiobject tracking using labeled multi-Bernoulli filters. The proposed framework is based on a novel approximation of the Cauchy-Schwarz divergence between the labeled multi-Bernoulli prior and posterior densities, which does not need Monte Carlo sampling of random sets in the multiobject space. The void probability functional is also formulated for labeled multi-Bernoulli distributions and used within our proposed method to form a constrained sensor control solution. Numerical studies demonstrate that reasonably acceptable movements are decided for the controlled sensor by our sensor control method, with the advantage that the void probability constraint is formally considered as part of the sensor control optimization algorithm.

Identifying the underlying model in a set of data contaminated by noise and outliers is a fundamental task in computer vision. The cost function associated with such tasks is often highly complex, hence in most cases only an approximate solution is obtained by evaluating the cost function on discrete locations in the parameter (hypothesis) space. To be successful at least one hypothesis has to be in the vicinity of the solution. Due to noise hypotheses generated by minimal subsets can be far from the underlying model, even when the samples are from the said structure. In this paper we investigate the feasibility of using higher than minimal subset sampling for hypothesis generation. Our empirical studies showed that increasing the sample size beyond minimal size ( p ), in particular up to p+2, will significantly increase the probability of generating a hypothesis closer to the true model when subsets are selected from inliers. On the other hand, the probability of selecting an all inlier sample rapidly decreases with the sample size, making direct extension of existing methods unfeasible. Hence, we propose a new computationally tractable method for robust model fitting that uses higher than minimal subsets. Here, one starts from an arbitrary hypothesis (which does not need to be in the vicinity of the solution) and moves until either a structure in data is found or the process is re-initialized. The method also has the ability to identify when the algorithm has reached a hypothesis with adequate accuracy and stops appropriately, thereby saving computational time. The experimental analysis carried out using synthetic and real data shows that the proposed method is both accurate and efficient compared to the state-of-the-art robust model fitting techniques.

Three‐dimensional (3D) motion analysis of dynamic body organs using volumetric images is of increasing interest in different computer vision applications. A number of methods for estimation of 3D optical flow in those images have been developed in recent years. However, theoretical limits of 3D optical flow‐based motion estimation and segmentation are yet to be analysed. In this study, a statistical analysis of 3D optical flow is presented and the results are used to predict the separability of local 3D motions. Experimental results, using both synthetic and real images, demonstrate the applicability of the proposed analysis to predict the separability of two motions in terms of the parameters quantifying their relative motion and the scale of measurement noise.

In this paper, we present a real time 3D SLAM system for texture-less scenes using only the depth information provided by a low cost RGB-D sensor. The proposed registration method is based on a novel informative sampling scheme that is able to extract the points carrying the most useful information from two consecutive frames. Those points are assigned 3D feature descriptors and matched with their correspondences from the other frame. A robust estimator is then used to refine the matches and estimate a rigid transformation between the frames. A global pose for the camera and associated 3D position of the points are computed by concatenating the relative transformations and a map is constructed. As a result, we are able to achieve very accurate registration in real-time using the sparse keypoints. Experimental evaluations show that our proposed method outperforms the state of the art registration algorithms in terms of accuracy and efficiency.

Various computer‐vision applications involve estimation of multiple motions from images of dynamic scenes. The exact nature of 3D‐object motions and the camera parameters are often not known a priori and therefore, the most general motion model (fundamental matrix) is applied. Although the estimation of fundamental matrix and its use for motion segmentation are established, the conditions for segmentation of different types of motions are largely unaddressed. In this study, we analysed the feasibility of motion segmentation using affine‐fundamental matrix, focusing on a scene includes multiple planar‐motions, viewed by an uncalibrated camera. We show that the successful segmentation of planar motion depends on several scene and motion parameters. Conditions to guarantee successful segmentation are proposed via extensive experiments using synthetic images. Experiments using real‐image data were set up to examine the relevance of those conditions to the scenarios in real applications. The experimental results demonstrate the capability of the proposed conditions to correctly predict the outcome of several segmentation scenarios and show the relevance of those conditions in real applications. In practice, the success of motion segmentation could be predicted from obtainable scene and motion parameters. Therefore these conditions serve as a guideline for practitioners in designing motion‐segmentation solutions.

Nonrigid image registration techniques using intensity based similarity measures are widely used in medical imaging applications. Due to high computational complexities of these techniques, particularly for volumetric images, finding appropriate registration methods to both reduce the computation burden and increase the registration accuracy has become an intensive area of research. In this paper, we propose a fast and accurate nonrigid registration method for intra-modality volumetric images. Our approach exploits the information provided by an order statistics based segmentation method, to find the important regions for registration and use an appropriate sampling scheme to target those areas and reduce the registration computation time. A unique advantage of the proposed method is its ability to identify the point of diminishing returns and stop the registration process. Our experiments on registration of end-inhale to end-exhale lung CT scan pairs, with expert annotated landmarks, show that the new method is both faster and more accurate than the state of the art sampling based techniques, particularly for registration of images with large deformations.

Measurement of local differences in the 3D motions of dynamic body organs (captured by volumetric scanners) is of increasing interest in biomedicai imaging applications. Estimation methods of 3D optical flow in those images have been studied in recent years. However, theoretical limits of 3D optical flow-based motion estimation and segmentation are yet to be analysed. In this paper, a novel criterion is proposed to statistically predict the separability of local 3D motions. Simulation results demonstrate how the proposed approach works in principle to predict separability of two motions in terms of the amount of relative motion and the scale of noise.

Wheel speed sensor is commonly used in conventional Antilock Braking Systems (ABS) to measure the wheel rotational speed. This paper proposes a novel architecture for In-Wheel Electric Vehicles (EVs) based on data fusion concept that provides improved accuracy, reliability, redundancy and fault-tolerance for the wheel speed measurement system of the ABS. The proposed method was extensively evaluated using actual ABS hardware. The experimental results showed that the accuracy of the wheel speed estimation based on the proposed data fusion were improved by up to 46 percent in comparison with a commercial ABS sensor. The proposed approach has the potential to substantially improve the performance of the ABS for In-Wheel EVs.

This letter addresses the sensor selection problem for tracking multiple dynamic targets within a sensor network. Since the bandwidth and energy of the sensor network are constrained, it would not be feasible to directly use the entire information of sensor nodes for detection and tracking of the targets and hence the need for sensor selection. Our sensor selection solution is formulated using the multi-Bernoulli random finite set framework. The proposed method selects a minimum subset of sensors which are most likely to provide reliable measurements. The overall scheme is a robust method that works in challenging scenarios where no prior information are available on clutter intensity or sensor detection profile. Simulation results demonstrate successful sensor selection in a challenging case where five targets move in a close vicinity to each other. Comparative results show the superior performance of our method in terms of accuracy of estimating the number of targets and their states.

Complexities of dynamic volumetric imaging challenge the available computer vision techniques on a number of different fronts. This paper examines the relationship between the estimation accuracy and required amount of smoothness for a general solution from a robust statistics perspective. We show that a (surprisingly) small amount of local smoothing is required to satisfy both the necessary and sufficient conditions for accurate optic flow estimation. This notion is called "just enough" smoothing, and its proper implementation has a profound effect on the preservation of local information in processing 3D dynamic scans. To demonstrate the effect of "just enough" smoothing, a robust 3D optic flow method with quantized local smoothing is presented, and the effect of local smoothing on the accuracy of motion estimation in dynamic lung CT images is examined using both synthetic and real image sequences with ground truth.

Elimination of a clamp force sensor from brake-by-wire system designs is strongly demanded due to implementation difficulties and cost issues. In this paper a new method is presented to estimate clamp force based on other sensory information. The proposed estimator fuses the outputs of two models to optimise the root mean square error (RMSE) of estimation. Experimental results show that the estimator can accurately track the true clamp force for high speed cases as demanded by anti-lock braking system (ABS) controls. A training strategy has been used to ensure that the estimator can successfully adapt to frictional variations within the reduction gearing. This paper is concluded with a discussion on the reliability of the developed clamp force estimator

In this paper, we address the problem of recovering the true underlying model of a surface while performing the segmentation. First, and in order to solve the model selection problem, we introduce a novel criterion, which is based on minimising strain energy of fitted surfaces. We then evaluate its performance and compare it with many other existing model selection techniques. Using this criterion, we then present a robust range data segmentation algorithm capable of segmenting complex objects with planar and curved surfaces. The presented algorithm simultaneously identifies the type (order and geometric shape) of each surface and separates all the points that are part of that surface. This paper includes the segmentation results of a large collection of range images obtained from objects with planar and curved surfaces. The resulting segmentation algorithm successfully segments various possible types of curved objects. More importantly, the new technique is capable of detecting the association between separated parts of a surface, which has the same Cartesian equation while segmenting a scene. This aspect is very useful in some industrial applications of range data analysis.

This paper proposes a robust estimator which is capable of segmenting multi-structural data. The proposed estimation technique is used to segment range data into linear (planar) and quadratic surfaces. The performance of the proposed range segmentation algorithm is tested on a number of real data sets.

A method for calculating optic flow, using robust statistics, is developed. The method generally out-performs all competing methods in terms of accuracy. One of the key features in the success of this method, is that we use least median of squares, which is known to be robust to outliers. The computational cost is kept very low by using an approximate solution to the least median of squares only in a first stage that detects outliers. The essential ingredients of our method should be applicable in a wide range of other computer vision problems.

A new approach to optic flow calculation, based on a highly robust statistical technique, is presented. In this algorithm, the optic flow problem is first formulated as a standard least squares problem. Then, its associated closest point problem is introduced and the transformation which takes this problem to a standard regression problem is provided. The least median of squares technique is used to solve the resulting regression problem. Some experimental results for both synthetic and real image sequences are also presented.