The paper presents an innovative non-destructive inspection (NDI) method, via employing a custom vision-based machine learning model, for evaluating the quality of Self-Piercing Rivet (SPR) joints, a commonly used joining technique in light vehicle production. Non-destructive inspection (NDI) is crucial for structural integrity analyses, including fracture and fatigue assessment. Utilizing μ-CT (Micro-Computed Tomography) scans of SPR joints and the Machine Vision AI (artificial intelligence) model, the present approach enables the detection and quantification of key quality parameters. The AI model is trained on images extracted from a comprehensive dataset of μ-CT scans of modified SPR joints. Samples used for training are modified carefully to represent the combined defects commonly found in SPR joints. The trained vision-based AI model is deployed to identify and quantify defects, accounting for cumulative errors from material inconsistencies, manufacturing imperfections, and dimensional tolerances. These parameters comprise cumulative defects that may affect joint integrity.The proposed method provides valuable cumulative measured inputs to numerical models of whole riveted structure, non-destructively, which are essential for evaluating the performance of complex structures incorporating SPRs. By significantly enhancing the prediction of the effects of manufacturing defects on lightweight structures, including fatigue life and joint durability, this approach enables more accurate and reliable assessments of individual joints; unlike current techniques, which depend on assumed, averaged, or estimated defect parameters without direct inspection. Ultimately, the AI-based NDI tool enables the enhancement of error detection precision, contributing to safer and more reliable structural integrity evaluations for a variety of manufacturing applications.

Domain adaptation is a well-studied field in machine learning. Distribution shift between train (source) and test (target) datasets is a common problem encountered in machine learning applications. One approach to resolve this issue is to use the Unsupervised Domain Adaptation (UDA) technique that carries out knowledge transfer from a label-rich source domain to an unlabeled target domain. Outliers that exist in either source or target datasets can introduce additional challenges when using UDA in practice. In this article, \(\alpha\) -divergence is used as a measure to minimize the discrepancy between the source and target distributions while inheriting robustness, adjustable with a single parameter \(\alpha\) , as the prominent feature of this measure. Here, it is shown that the other well-known divergence-based UDA techniques can be derived as special cases of the proposed method. Furthermore, a theoretical upper bound is derived for the loss in the target domain in terms of the source loss and the \(\alpha\) -divergence between the joint distributions in the two domains. The robustness of the proposed method is validated through testing on several benchmarked datasets in open-set and partial UDA setups where extra classes existing in target and source datasets are considered as outliers. The code is publicly available at https://github.com/rashidis/IT-RUDA .

Advanced machine vision and deep learning models are now widely used as virtual sensors in various applications to detect and classify objects in image measurements. Typically, these virtual sensors output measurements as a set of class probabilities for each detected object within the sensor’s field of view. However, integrating this type of data into multi-target tracking systems, traditionally based on point measurement detections, presents some challenges. This paper proposes a solution by introducing a labeled multi-Bernoulli filter formulated to handle detections with class measurements from virtual sensors for multi-target tracking. Furthermore, we explore the application of this filter for multi-sensor multi-target tracking within a distributed sensor network. Through numerical experiments involving vehicles equipped with cameras and deep learning classification modules navigating complex roads, we demonstrate that incorporating class information into the tracking process improves tracking accuracy. This improvement is further enhanced when vehicles share and fuse their information in a distributed manner. Our findings highlight the benefits of integrating class probability data into multi-target tracking filters, offering substantial improvements in the accuracy of automated monitoring systems in dynamic and complex environments.

Deep learning techniques often perform poorly in the presence of domain shift, where the test data follows a different distribution than the training data. The most practically desirable approach to address this issue is Single Domain Generalization (S-DG), which aims to train robust models using data from a single source. Prior work on S-DG has primarily focused on using data augmentation techniques to generate diverse training data. In this paper, we explore an alternative approach by investigating the robustness of linear operators, such as convolution and dense layers commonly used in deep learning. We propose a novel operator called "XCNorm" that computes the normalized cross-correlation between weights and an input feature patch. This approach is invariant to both affine shifts and changes in energy within a local feature patch and eliminates the need for commonly used non-linear activation functions. We show that deep neural networks composed of this operator are robust to common semantic distribution shifts. Furthermore, our empirical results on single-domain generalization benchmarks demonstrate that our proposed technique performs comparably to the stateof-the-art methods.1

In this paper, we propose a lightweight approach for industrial defect detection that is modeled based on anomaly detection using point pattern data. Most recent works use global features for feature extraction to summarize image content. However, global features are not robust against lighting and viewpoint changes and do not describe images’ geometrical information which, thus, cannot be fully utilized in manufacturing industry. To the best of our knowledge, we are the first to propose using transfer learning of local/point pattern features to overcome these limitations and capture geometrical information of the interested image regions. We model these local/point pattern features as a random finite set (RFS). In addition, we propose RFS energy, in contrast to RFS likelihood as an anomaly score. The similarity distribution of point pattern features of the normal sample has been modeled as a multivariate Gaussian. Parameters learning of the proposed RFS energy does not require any heavy computation. We evaluate the proposed approach on the MVTec AD dataset, a multi-object defect detection dataset. Experimental results show the outstanding performance of our proposed approach compared to the state-of-the-art methods, and the proposed RFS energy outperforms the state-of-the-art in the few shot learning settings. Also, we evaluated the proposed approach on needle dataset, a real life application, to detect defective needle samples given three different views. The results show superior performance compared to DiffeNet.

This paper introduces an approach to distributed multi-object tracking for connected vehicles, aiming to overcome the inherent challenges of label inconsistency and double counting prevalent in distributed information fusion methods, particularly in the context of situational awareness for connected vehicles. Our proposed method expands the label space by incorporating sensor identity into the object’s label. Furthermore, we present an intuitive merging algorithm designed to effectively eliminate instances of double counting. The approach is formulated, and an algorithm is developed for implementation within a labeled multi-Bernoulli filter, executed locally on each node of a distributed network responsible for information fusion. To assess the efficacy of our solution, we evaluate its performance in a highly demanding scenario specifically designed for intelligent transport systems and compare its performance against alternative approaches.

We present a framework for the visualization of mechanical stress using augmented reality (AR) using Thermoelastic Stress Analysis (TSA). The 2D stress images generated by TSA are converted to a 3D stress map using computer vision technology and then superimposed on the real object using AR. Our framework enables in-situ visualization of stress in geometrically complex structural components, which can assist in the design, manufacture, test, and through-life sustainment of failure-critical engineering assets. We also discuss the challenges of such a TSA-AR combination and present a case study that demonstrates the performance and significance of our system.

Despite significant amount of work reported in the computer vision literature, segmenting images or videos based on multiple cues such as objectness, texture and motion, is still a challenge. This is particularly true when the number of objects to be segmented is not known or there are objects that are not classified in the training data (unknown objects). A possible remedy to this problem is to utiize graph-based clustering techniques such as Correlation Clustering. It is known that using long range affinities (Lifted multicut), makes correlation clustering more accurate than using only adjacent affinities (Multicut). However, the former is computationally expensive and hard to use. In this paper, we introduce a new framework to perform image/motion segmentation using an affinity learning module and a Message Passing Graph Neural Network (MPGNN). The affinity learning module uses a permutation invariant affinity representation to overcome the multi-object problem. The paper shows, both theoretically and empirically, that the proposed MPGNN aggregates higher order information and thereby converts the Lifted Multicut Problem (LMP) to a Multicut Problem (MP), which is easier and faster to solve. Importantly, the proposed method can be generalized to deal with different clustering problems with the same MPGNN architecture. For instance, our method produces competitive results for single image segmentation (on BSDS dataset) as well as unsupervised video object segmentation (on DAVIS17 dataset), by only changing the feature extraction part. In addition, using an ablation study on the proposed MPGNN architecture, we show that the way we update the parameterized affinities directly contributes to the accuracy of the results.

Deep convolutional neural networks for dense prediction tasks are commonly optimized using synthetic data, as generating pixel-wise annotations for real-world data is laborious. However, the synthetically trained models do not generalize well to real-world environments. This poor "synthetic to real" (S2R) generalization we address through the lens of shortcut learning. We demonstrate that the learning of feature representations in deep convolutional networks is heavily influenced by synthetic data artifacts (shortcut attributes). To mitigate this issue, we propose an Information-Theoretic Shortcut Avoidance (ITSA) approach to automatically restrict shortcut-related information from being encoded into the feature representations. Specifically, our proposed method minimizes the sensitivity of latent features to input variations: to regularize the learning of robust and shortcut-invariant features in synthetically trained models. To avoid the prohibitive computational cost of direct input sensitivity optimization, we propose a practical yet feasible algorithm to achieve robustness. Our results show that the proposed method can effectively improve S2R generalization in multiple distinct dense prediction tasks, such as stereo matching, optical flow, and semantic segmentation. Importantly, the proposed method enhances the robustness of the synthetically trained networks and outperforms their fine-tuned counterparts (on real data) for challenging out-of-domain applications.

Tracking multiple objects through time is an important part of an intelligent transportation system. Random finite set (RFS)-based filters are one of the emerging techniques for tracking multiple objects. In multi-object tracking (MOT), a common assumption is that each object is moving independent of its surroundings. However, in many real-world applications, objects interact with one another and the environment. Such interactions are rarely considered within the tracking process. In this paper, we present a novel approach to incorporate target interactions within the prediction step of a RFS-based multi-target filter, i.e. labeled multi-Bernoulli (LMB) filter. The proposed method is capable of explicitly incorporating the effect of target interactions within the filtering process, with little to no changes made for specific applications. We have developed the proposed filter for two practical applications, i.e. tracking a coordinated swarm and vehicles. The method has been tested for a complex vehicle tracking dataset and compared with the PHD filter and LMB filter through the OSPA metric, and with the LMB filter through OSPA(2) metric and cardinality error. The results demonstrate that the proposed interaction-aware method depicts considerable performance enhancement over the other methods in terms of the selected metrics.

State-of-the-art stereo matching networks trained only on synthetic data often fail to generalize to more challenging real data domains. In this paper, we attempt to unfold an important factor that hinders the networks from generalizing across domains: through the lens of shortcut learning. We demonstrate that the learning of feature representations in stereo matching networks is heavily influenced by synthetic data artefacts (shortcut attributes). To mitigate this issue, we propose an Information-Theoretic Shortcut Avoidance (ITSA) approach to automatically restrict shortcut-related information from being encoded into the feature representations. As a result, our proposed method learns robust and shortcut-invariant features by minimizing the sensitivity of latent features to input variations. To avoid the prohibitive computational cost of direct input sensitivity optimization, we propose an effective yet feasible algorithm to achieve robustness. We show that using this method, state-of-the-art stereo matching networks that are trained purely on synthetic data can effectively generalize to challenging and previously unseen real data scenarios. Importantly, the proposed method enhances the robustness of the synthetic trained networks to the point that they outperform their fine-tuned counterparts (on real data) for challenging out-of-domain stereo datasets.

In autonomous vehicles, depth information for the environment surrounding the vehicle is commonly extracted using time-of-flight (ToF) sensors such as LiDARs and RADARs. Those sensors have some limitations that may potentially degrade the quality and utility of the depth information to a substantial extent. An alternative solution is depth estimation from stereo pairs. However, stereo matching and depth estimation often fails at ill-posed regions including areas with repetitive patterns or textureless surfaces which are commonly found on planar surfaces. This paper focuses on designing an efficient framework for stereo depth estimation, using deep learning technique, that is robust against the mentioned ill-posed regions. With the observation that disparities of all pixels belonging to planar areas (scene plane) viewed by two rectified stereo images can be described using affine transformations, our proposed method predicts pixel-wise affine transformation parameters based on the depth information encoded in the aggregated cost volume. We also introduce a propagation term which enforces all pixels belonging to the same scene plane to be transformed using the same parameters. Disparity can then be computed by multiplying the predicted affine parameters with the corresponding pixel locations. The proposed method was evaluated on several benchmark datasets. We are able to obtain competitive results and at the same time reducing the processing time of common convolution neural network (CNN) in stereo matching by 50%. Analysis of the findings shows that our method can produce reliable results at the ill-posed regions which are challenging to the current state-of-the-arts methods.

Autonomous vehicles in intelligent transportation systems must be able to perform reliable and safe navigation. This necessitates accurate object detection, which is commonly achieved by high-precision depth perception. Existing stereo vision-based depth estimation systems generally involve computation of pixel correspondences and estimation of disparities between rectified image pairs. The estimated disparity values will be converted into depth values in downstream applications. As most applications often work in the depth domain, the accuracy of depth estimation is often more compelling than disparity estimation. However, at large distances (> 50m), the accuracy of disparity estimation does not directly translate to the accuracy of depth estimation. In the context of learning-based stereo systems, this is mainly due to biases imposed by the choices of the disparity-based loss function and the training data. Consequently, the learning algorithms often produce unreliable depth estimates of under-represented foreground objects, particularly at large distances. To resolve this issue, we first analyze the effect of those biases and then propose a pair of depth-based loss functions for foreground objects and background separately. These loss functions can be tuned and can balance the inherent bias of the stereo learning algorithms. The efficacy of our solution is demonstrated by an extensive set of experiments, which are benchmarked against state of the art. We show on the KITTI 2015 benchmark that our proposed solution yields substantial improvements in disparity and depth estimation, particularly for objects located at distances beyond 50 meters, outperforming the previous state of the art by 10%.

Proactive inspection is essential for prediction and prevention of rolling stock component failures. The conventional process for inspecting bogies under trains presents significant challenges for inspectors who need to visually check the tight and cluttered environment. We propose a miniature multi-link climbing robot, called BogieBot, that can be deployed inside the undercarriage areas of trains and other large vehicles for inspection and maintenance purposes. BogieBot can carry a visual sensor or manipulator on its main body. The novel compact design utilises six identical couple joints and two mechanically switchable magnetic grippers that together, empower multi-modal climbing and manipulation. The proposed mechanism is kinematically redundant, allowing the robot to perform self-motions in a tight space and manoeuvre around obstacles. The mechanism design and various analyses on the forward and inverse kinematic, work-space, and self-motions of BogieBot are presented. The robot is demonstrated to perform challenging navigation tasks in different scenarios involving simulated complex environments.

This paper presents an innovative method for motion segmentation in RGB-D dynamic videos with multiple moving objects. The focus is on finding static, small or slow moving objects (often overlooked by other methods) that their inclusion can improve the motion segmentation results. In our approach, semantic object based segmentation and motion cues are combined to estimate the number of moving objects, their motion parameters and perform segmentation. Selective object-based sampling and correspondence matching are used to estimate object specific motion parameters. The main issue with such an approach is the over segmentation of moving parts due to the fact that different objects can have the same motion (e.g. background objects). To resolve this issue, we propose to identify objects with similar motions by characterizing each motion by a distribution of a simple metric and using a statistical inference theory to assess their similarities. To demonstrate the significance of the proposed statistical inference, we present an ablation study, with and without static objects inclusion, on SLAM accuracy using the TUM-RGBD dataset. To test the effectiveness of the proposed method for finding small or slow moving objects, we applied the method to RGB-D MultiBody and SBM-RGBD motion segmentation datasets. The results showed that we can improve the accuracy of motion segmentation for small objects while remaining competitive on overall measures.

Volumetric imaging is an essential diagnostic tool for medical practitioners. The use of popular techniques such as convolutional neural networks (CNN) for analysis of volumetric images is constrained by the availability of detailed (with local annotations) training data and GPU memory. In this paper, the volumetric image classification problem is posed as a multi-instance classification problem and a novel method is proposed to adaptively select positive instances from positive bags during the training phase. This method uses the extreme value theory to model the feature distribution of the images without a pathology and use it to identify positive instances of an imaged pathology. The experimental results, on three separate image classification tasks (i.e. classify retinal OCT images according to the presence or absence of fluid build-ups, emphysema detection in pulmonary 3D-CT images and detection of cancerous regions in 2D histopathology images) show that the proposed method produces classifiers that have similar performance to fully supervised methods and achieves the state of the art performance in all examined test cases.

Classification of 3D shapes into physically meaningful categories is one of the most important tasks in understanding the immediate environment. Methods that leverage the recent advancements in deep learning have shown to outperform the traditional approaches. However, performances of those methods have only been analyzed with relatively clean data. Three-dimensional measurement sets (point clouds) produced by 3D scanners are rarely that accurate and often contain noise, outliers or missing points. This paper presents an extensive analysis of the robustness of the state-of-the-art neural network algorithms to realistic data inaccuracies. Our experiments show that the existence of these inaccuracies can significantly affect the performance of the deep learning-based algorithms.

In this paper we present a novel non-Bayesian filtering method for tracking multiple objects with a particular application in time-lapse cell microscopic video sequence. In our method the heat-map of the frame sequence is extracted and represented as a pseudo-probability hypothesis density of the image. The pseudo-probability hypothesis density is used as measurements and fused with a prior Poisson random finite set density. We employed Cauchy-Schwarz divergence for information fusion. The presented algorithm was tested on a publicly available cell microscopic video sequence.

Cell life-cycle and motility analysis is one of the fundamental tasks in many biological research activities, and its automation is a challenging problem. To solve the cell tracking problem using a Bayesian stochastic filter, one needs to properly incorporate all the information available about the cell behavior within the filter. This includes its movements, changes during mitosis process (up to splitting) and death. This paper demonstrates an effective way to perform this task for a particular family of cells (Chinese Hamster Ovarian (CHO) cells) that are known to have a commonly elliptic shape, and elongate during their mitosis. We model this by incorporating an ellipse-based state variable, a particular spawning process and an intuitive adaptive (measurement-driven) birth process that are added to the prediction step of a multi-Bernoulli filter. Our numerical experimental results involving microscopic images of living cells demonstrate significant improvement in tracking performance as a result of the proposed additions.

With the recent advent of labeled random finite set filters, it is now possible to not only estimate the number of objects and their states, but also track their trajectories all within the stochastic filtering scheme. This paper investigates how the objects label information returned by a labeled multi-Bernoulli filter can be effectively used for sensor control purposes. The main focus is on selective multi-object tracking applications where objects with particular labels are of high priority, and the sensor needs to be controlled to achieve maximum confidence in tracking performance of the filter for those objects. We formulated and examined two novel solutions. The first is based on maximization of the confidence in what the filter returns in relation to the existence of the objects of interest. The second solution is based on maximization of the confidence in what the filter returns in relation to the both existence and the states of the objects of interest. We also present an intuitive solution for tracking scenarios when some of the targets of interest temporarily disappear then reappear. Simulation results indicate how the proposed methods can lead to significant improvements in terms of tracking accuracy of objects of interest, compared to using the generic (non-selective) sensor control methods.

Condition monitoring of storm-water pipe systems are carried-out regularly using semi-automated processors. Semi-automated inspection is time consuming, expensive and produces varying and relatively unreliable results due to operators fatigue and novicity. This paper propose an innovative method to automate the storm-water pipe inspection and condition assessment process which employs a computer vision algorithm based on deep-neural network architecture to classify the defect types automatically. With the proposed method, the operator only needs to guide the robot through each pipe and no longer needs to be an expert. The results obtained on a CCTV video dataset of storm-water pipes shows that the deep neural network architectures trained with data augmentation and transfer learning is capable of achieving high accuracies in identifying the defect types.

This paper approaches the problem of geometric multi-model fitting as a data segmentation problem. The proposed solution is based on a sequence of sampling hyperedges from a hypergraph, model selection and hypergraph clustering steps. We developed a sampling method that significantly facilitates solving the segmentation problem using a new form of the Markov-Chain-Monte-Carlo (MCMC) method to effectively sample from hyperedge distribution. To sample from this distribution effectively, our proposed Markov Chain includes new ways of long and short jumps to perform exploration and exploitation of all structures. To enhance the quality of samples, a greedy algorithm is used to exploit nearby structure based on the minimization of the Least k th Order Statistics cost function . Unlike common sampling methods, ours does not require any specific prior knowledge about the distribution of models. The output set of samples leads to a clustering solution by which the final model parameters for each segment are obtained. The method competes favorably with the state-of-the-art both in terms of computation power and segmentation accuracy.

Identifying the underlying models in a set of data points that is contaminated by noise and outliers leads to a highly complex multi-model fitting problem. This problem can be posed as a clustering problem by the projection of higher-order affinities between data points into a graph, which can be clustered using spectral clustering. Calculating all possible higher-order affinities is computationally expensive. Hence, in most cases, only a subset is used. In this paper, we propose an effective sampling method for obtaining a highly accurate approximation of the full graph, which is required to solve multi-structural model fitting problems in computer vision. The proposed method is based on the observation that the usefulness of a graph for segmentation improves as the distribution of the hypotheses that are used to build the graph approaches the distribution of the actual parameters for the given data. In this paper, we approximate this actual parameter distribution by using a th-order statistics-based cost function, and the samples are generated using a greedy algorithm that is coupled with a data sub-sampling strategy. The experimental analysis shows that the proposed method is both accurate and computationally efficient compared with the state-of-the-art robust multi-model fitting techniques. The implementation of the method is publicly available from https://github.com/RuwanT/model-fitting-cbs.

A constrained sensor control method is presented for multiobject tracking using labeled multi-Bernoulli filters. The proposed framework is based on a novel approximation of the Cauchy-Schwarz divergence between the labeled multi-Bernoulli prior and posterior densities, which does not need Monte Carlo sampling of random sets in the multiobject space. The void probability functional is also formulated for labeled multi-Bernoulli distributions and used within our proposed method to form a constrained sensor control solution. Numerical studies demonstrate that reasonably acceptable movements are decided for the controlled sensor by our sensor control method, with the advantage that the void probability constraint is formally considered as part of the sensor control optimization algorithm.

Identifying the underlying model in a set of data contaminated by noise and outliers is a fundamental task in computer vision. The cost function associated with such tasks is often highly complex, hence in most cases only an approximate solution is obtained by evaluating the cost function on discrete locations in the parameter (hypothesis) space. To be successful at least one hypothesis has to be in the vicinity of the solution. Due to noise hypotheses generated by minimal subsets can be far from the underlying model, even when the samples are from the said structure. In this paper we investigate the feasibility of using higher than minimal subset sampling for hypothesis generation. Our empirical studies showed that increasing the sample size beyond minimal size ( p ), in particular up to p+2, will significantly increase the probability of generating a hypothesis closer to the true model when subsets are selected from inliers. On the other hand, the probability of selecting an all inlier sample rapidly decreases with the sample size, making direct extension of existing methods unfeasible. Hence, we propose a new computationally tractable method for robust model fitting that uses higher than minimal subsets. Here, one starts from an arbitrary hypothesis (which does not need to be in the vicinity of the solution) and moves until either a structure in data is found or the process is re-initialized. The method also has the ability to identify when the algorithm has reached a hypothesis with adequate accuracy and stops appropriately, thereby saving computational time. The experimental analysis carried out using synthetic and real data shows that the proposed method is both accurate and efficient compared to the state-of-the-art robust model fitting techniques.

Three‐dimensional (3D) motion analysis of dynamic body organs using volumetric images is of increasing interest in different computer vision applications. A number of methods for estimation of 3D optical flow in those images have been developed in recent years. However, theoretical limits of 3D optical flow‐based motion estimation and segmentation are yet to be analysed. In this study, a statistical analysis of 3D optical flow is presented and the results are used to predict the separability of local 3D motions. Experimental results, using both synthetic and real images, demonstrate the applicability of the proposed analysis to predict the separability of two motions in terms of the parameters quantifying their relative motion and the scale of measurement noise.

In this paper, we present a real time 3D SLAM system for texture-less scenes using only the depth information provided by a low cost RGB-D sensor. The proposed registration method is based on a novel informative sampling scheme that is able to extract the points carrying the most useful information from two consecutive frames. Those points are assigned 3D feature descriptors and matched with their correspondences from the other frame. A robust estimator is then used to refine the matches and estimate a rigid transformation between the frames. A global pose for the camera and associated 3D position of the points are computed by concatenating the relative transformations and a map is constructed. As a result, we are able to achieve very accurate registration in real-time using the sparse keypoints. Experimental evaluations show that our proposed method outperforms the state of the art registration algorithms in terms of accuracy and efficiency.

Various computer‐vision applications involve estimation of multiple motions from images of dynamic scenes. The exact nature of 3D‐object motions and the camera parameters are often not known a priori and therefore, the most general motion model (fundamental matrix) is applied. Although the estimation of fundamental matrix and its use for motion segmentation are established, the conditions for segmentation of different types of motions are largely unaddressed. In this study, we analysed the feasibility of motion segmentation using affine‐fundamental matrix, focusing on a scene includes multiple planar‐motions, viewed by an uncalibrated camera. We show that the successful segmentation of planar motion depends on several scene and motion parameters. Conditions to guarantee successful segmentation are proposed via extensive experiments using synthetic images. Experiments using real‐image data were set up to examine the relevance of those conditions to the scenarios in real applications. The experimental results demonstrate the capability of the proposed conditions to correctly predict the outcome of several segmentation scenarios and show the relevance of those conditions in real applications. In practice, the success of motion segmentation could be predicted from obtainable scene and motion parameters. Therefore these conditions serve as a guideline for practitioners in designing motion‐segmentation solutions.

Measurement of local differences in the 3D motions of dynamic body organs (captured by volumetric scanners) is of increasing interest in biomedicai imaging applications. Estimation methods of 3D optical flow in those images have been studied in recent years. However, theoretical limits of 3D optical flow-based motion estimation and segmentation are yet to be analysed. In this paper, a novel criterion is proposed to statistically predict the separability of local 3D motions. Simulation results demonstrate how the proposed approach works in principle to predict separability of two motions in terms of the amount of relative motion and the scale of noise.

This letter addresses the sensor selection problem for tracking multiple dynamic targets within a sensor network. Since the bandwidth and energy of the sensor network are constrained, it would not be feasible to directly use the entire information of sensor nodes for detection and tracking of the targets and hence the need for sensor selection. Our sensor selection solution is formulated using the multi-Bernoulli random finite set framework. The proposed method selects a minimum subset of sensors which are most likely to provide reliable measurements. The overall scheme is a robust method that works in challenging scenarios where no prior information are available on clutter intensity or sensor detection profile. Simulation results demonstrate successful sensor selection in a challenging case where five targets move in a close vicinity to each other. Comparative results show the superior performance of our method in terms of accuracy of estimating the number of targets and their states.

A new method is presented for integration of audio and visual information in multiple target tracking applications. The proposed approach uses a Bayesian filtering formulation and exploits multi-Bernoulli random finite set approximations. The work presented in this paper is the first principled Bayesian estimation approach to solve the sensor fusion problems that involve intermittent sensory data (e.g. audio data for a person who occasionally speaks.) We have examined our method with case studies from the SPEVI database. The results show nearly perfect tracking of people not only when they are silent but also when they are not visible to the camera (but speaking).

Almost every single-view visual multi-target tracking method presented in the literature includes a detection routine that maps the image data to point measurements relevant to the target states. These measurements are commonly further processed by a filter to estimate the number of targets and their states. This paper presents a novel visual tracking technique based on a multi-object filtering algorithm that operates directly on the image observations without the need for any detection. Experimental results on tracking sport players show that our proposed method can automatically track numerous interacting targets and quickly finds players entering or leaving the scene.

Elimination of a clamp force sensor from brake-by-wire system designs is strongly demanded due to implementation difficulties and cost issues. In this paper a new method is presented to estimate clamp force based on other sensory information. The proposed estimator fuses the outputs of two models to optimise the root mean square error (RMSE) of estimation. Experimental results show that the estimator can accurately track the true clamp force for high speed cases as demanded by anti-lock braking system (ABS) controls. A training strategy has been used to ensure that the estimator can successfully adapt to frictional variations within the reduction gearing. This paper is concluded with a discussion on the reliability of the developed clamp force estimator