Video inpainting is attracting more attention due to the potential applications of video object removal and video content restoration. Current approaches either use end-to-end methods to generate missing pixels directly or perform indirect transfer for known regions based on motion field guidance. However, such approaches cannot handle both high-resolution images and diverse degrees of scene variation between adjacent video frames, and they cannot achieve clear and accurate inpainting effects for large continuous missing areas. To this end, we propose an adaptive multi-stage interval video inpainting algorithm guided by anisotropic optical flow. First, we customize an optical flow inpainting method guided by single image inpainting, enabling optical flow to maintain a strong self-healing ability over a large range of missing areas. Then, the interval mechanism adaptively determines the required temporal neighbors for missing pixels by assessing video attributes and inpainted optical flow results. After the missing pixels complete the multi-candidate information fusion in their associated temporal neighbors, we obtain spatio-temporally consistent and accurate results. Finally, extensive experiments on the YouTubeVOS, DAVIS, A2D2, and custom datasets show that our proposed approach has achieved state-of-the-art performance with good environmental migration ability.

In-situ monitoring of an Additive Manufacturing (AM) process is the way to enhance the quality of the components manufactured by it. However, the metal AM processes are complex to monitor because of usage of high energy-based heat source in melting the deposition material, particularly in the arc-based AM processes where arc and sparks make it difficult to capture the deposition. This paper explores the use of a High Dynamic Range (HDR) camera to capture and monitor deposition processes for μ-Plasma Transferred Arc Additive Manufacturing (μP-TAAM) process. Additionally, it proposes the YOLO-based object detection model to assess and monitor the quality of Co-Cr-Mo-4Ti depositions. The research focuses on analysing the performance of YOLOv8l, YOLOv9t, YOLOv9s, YOLOv9m and YOLOv10n models to detect and classify good and bad depositions. It has been found that YOLIv9m gave a strong balance across all evaluation metrics such as highest Recall of 0.983, high precision of 0.98, high mAP50 of 0.994 and high mAP50-95 of 0.848. These findings underscore the model's potential for deployment in in-situ monitoring scenarios.

This paper presents a groundbreaking advancement in industrial quality control through the development of an automated soldering quality assessment system for circuit boards utilizing hyperspectral imaging (USI) technology. Building upon the transformative capabilities of USI in visual inspection, our research focuses on enhancing the precision and depth of assessment in soldering processes, a critical aspect of electronics manufacturing. By leveraging the unique spectral information captured by HSI, beyond the capabilities of traditional vision systems, our automated solution offers a comprehensive evaluation of solder quality, overcoming challenges posed by similar absorption characteristics of materials. We detail the methodology, algorithms, and integration of HSI into the inspection pipeline, highlighting its effectiveness in detecting defects, ensuring uniformity, and improving overall product quality. The application of this technology extends beyond electronics manufacturing, with potential implications for various industries requiring meticulous quality control. Through this study, we contribute to the ongoing evolution of visual inspection systems, empowering industries with advanced tools for precise and reliable quality assessment.

In the fields of manufacturing and robotics, accurately determining the orientation of manufacturing components, such as bolts, is a critical yet challenging problem due to the limitations of existing detection methods. This study introduces a novel methodology for addressing this issue, leveraging traditional computer vision techniques, by proposing a streamlined approach that exploits the inherent geometric properties of bolts for orientation detection. Two methods are presented to ascertain the initial axis angle of the bolt: the Progressive Probabilistic Hough Transform (PPHT) and Principal Component Analysis (PCA). These methods are used in conjunction with a novel tip direction detection approach. The results of the study demonstrate consistent accuracy in angle determination, with PPHT and PCA both achieving angular deviations below ±0.5° in the simple dataset, and PCA showing enhanced robustness in the dataset degraded by shadows, with a maximum error under ±1.5°. This research not only reaffirms the viability of fundamental computer vision techniques in modern robotic applications but also sets a precedent for simple, generalisable, and reliable orientation detection solutions. These methods effectively bridge the gap between highly specialised machine learning systems, which often require tailored, complex models and extensive training data, and more universally applicable, straightforward approaches.

This paper presents two novel feature representations for neuromorphic audio data. Neuromorphic audio data are considered state-of-the-art when precise time responses are needed while also keeping energy-demands to a minimum. The approaches presented here are based on the concept of phased encoding of neuromorphic data to generate feature representations. One of the approaches enhances on this further by utilising an autoencoder to reduce the dimensionality of the feature representation allowing for increased accuracy in noise-rich environments such as industrial shop floors. The approaches are evaluated against other leading audio feature representation methods using a neuromorphic version of the TIDIGITS database and results demonstrate high accuracy for the proposed approach. We also find that the autoencoder-backed method achieves the best performance compared with the other methods as the dimensionality reduction results in a generalised representation of the feature set which is less sensitive when compared to other methods.

This paper explores the application of BlazePose, a monocular human pose estimation (HPE) model, within a teleoperation framework for a UR5 six-axis robot. Achieving teleoperation with only a single RGB camera and a device without a powerful GPU will improve accessibility and cost effectiveness of teleoperation solutions. This study evaluates the 2D pose estimation capabilities of BlazePose for robotic teleoperation tasks. Given the necessity of manipulating the UR5 in three- dimensional space, we implement a 2D-based controller that translates the teleoperator's 2D right hand position within a configurable hand workspace to the corresponding position of the robot's Tool Centre Point (TCP) within the robot's available workspace along two dimensions. The left hand is then utilised for controlling the robot's motion along the third dimension and operating the attached OnRobot RG2 gripper during the pick- and-place task. Additionally, we explore an alternative control paradigm utilising the 3D pose estimation of BlazePose for a more intuitive controller. Two experiments are conducted: the pick-and-place task to assess the 2D-based controller in common robotic tasks, and a hold position task. The hold position task aims to assess the amount of excess movement attributable to the HPE model when utilising the 3D-based controller. The results reveal that while the 2D pose estimation capabilities enable effective teleoperation, the utilisation of 3D estimation results in poor translation to robot control and significant excess motion. These findings underscore the importance of accurate depth estimation in 3D HPE models for precise and reliable teleoperation.

Improvements to human-robot interaction methods could increase the ease of use of robots in manufacturing environments. Many of these environments are noisy and therefore preclude the use of audio communication between humans or in human-robot interactions. Therefore, this paper proposes using a gesture based communication system for robot control. To that end, the VGG16 and VGG19 convolutional neural network (CNN) structures are used for gesture classification along with 3 datasets of American Sign Language (ASL) fingerspelling images. The model performance is evaluated, and modifications made to their parameters to improve performance, before applying them to robot control tasks. The results show that with parameter tuning, test accuracies of up to, 100% are achievable.

In semiconductor manufacturing industry, automated systems are essential for efficient and accurate identification of defects, prior to final product completion, to ensure quality and reduce waste. To achieve this, semiconductor industries are developing smart inspection systems to identify defects on the surface of wafers during manufacturing. Computer vision techniques play a crucial role in developing accurate inspection systems. However, most existing computer vision-based systems perform poorly when classifying defects, and many manufacturing companies still rely on manual inspection. To overcome this, we propose an efficient method for classifying defects in an industrial dataset using EfficientNet-B4 transfer learning along with Squeeze and Excitation block and multilayer perceptron. Furthermore, we utilise the various data-augmentation techniques to enrich the dataset and improve the generalisation of proposed model. This proposed method is lightweight and can classify defects in real-time with an accuracy of approximately 98%.

Cross-view geo-localization refers to recognizing images of the same geographic target obtained from different platforms (such as drone-view, satellite-view and ground-view). However, cross-view geo-localization is challenging as image capture using different platforms coupled with extreme viewpoint variations can cause significant changes to the visual image content. Existing methods mainly focus on mining the fine-grained features or the contextual information in neighboring areas, but ignore the complete information of the entire image and the association of contextual information of adjacent regions. Therefore, a multi-branch joint representation learning network model based on information fusion strategies is proposed to solve this cross-view geo-localization problem. Firstly, we obtain feature information from the image through global information fusion branch and local information fusion branch to help the network learn the discernable information in the different images. In addition, a local-guided-global information fusion branch is introduced to make local information assist global features to enhance the learning of potential information in the images. Secondly, we introduced different information fusion strategies in each branch to increase the extraction of contextual information through expanding the global receptive field, thus improving the performance of the model. Finally, a series of experiments is carried out on four prevailing benchmark datasets, namely University-1652, SUES-200, CVUAS and CVACT datasets. The quantitative comparisons from the experiments clearly indicate that the proposed network framework has great performance. For example, compared with some state-of-the-art methods, the quantitative improvements of the R@1 and AP on the University-1652 datasets are 1.91%, 2.18% and 1.55%, 2.99% in both tasks, respectively.

Cross-view geo-localization refers to the task of matching the same geographic target using images obtained from different platforms, such as drone-view and satellite-view. However, the view angle of images obtained through different platforms will vary greatly, which can bring great challenges to the cross-view geo-localization task. Therefore, we propose a multi-level feedback joint representation learning network based on adaptive area elimination to solve the cross-view geo-localization problem. In our network model, we first process the extracted global features to obtain part-level and patch-level features. We then utilize these features as feedback to the global features to extract the contextual information in the global features and improve the robustness of the extracted features. In addition, as images obtained from different platforms differ, there will always be some interference when matching images. Therefore, we introduce an adaptive area elimination strategy to erase the interference information in the global features and assist the model in obtaining crucial information. On this basis, the feature correlation loss function is designed to constrain learning when using global feature information, thereby eliminating the possible interference, which can improve the network model performance. Finally, a series of experiments is carried out using two well-known benchmarks, namely University-1652 and SUES-200, and the experimental results show that the proposed network model achieves competitive results, thereby demonstrating the effectiveness of proposed model.

Dynamic SLAM is a key technology for autonomous driving and robotics, and accurate pose estimation of surrounding objects is important for semantic perception tasks. Current quadric SLAM methods are based on the assumption of a static environment and can only reconstruct static quadrics in the scene, which limits their applications in complex dynamic scenarios. In this paper, we propose a visual SLAM system that is capable of reconstructing dynamic objects as quadrics, with a unified framework for jointly optimizing pose estimation, multi-object tracking (MOT), and quadric parameters. We propose a robust object-centric quadric initialization algorithm for both static and moving objects, which decouples the prior estimation of the object pose from the quadric parameters. The object is initialized with a coarse sphere, and quadric parameters are further refined. We design a novel factor graph that tightly optimizes camera pose, object pose, map points and quadric parameters within the sliding window-based optimization. To the best of our knowledge, we are the first to propose a dynamic SLAM that combines quadric representations and MOT in a tightly coupled optimization. We perform qualitative and quantitative experiments on both simulated and real-world datasets, and demonstrate the robustness and accuracy in terms of camera localization, dynamic quadric initialization, mapping and tracking. Our system demonstrates the potential application of object perception with quadric representation in complex dynamic scenes.

Simultaneous ego localization and surrounding object motion awareness are significant issues for the navigation capability of unmanned systems and virtual-real interaction applications. Robust and accurate data association at object and feature levels is one of the key factors in solving this problem. However, currently available solutions ignore the complementarity among different cues in the front-end object association and the negative effects of poorly tracked features on the back-end optimization. It makes them not robust enough in practical applications. Motivated by these observations, we make up rigid environment as a unified whole to assist state decoupling by integrating high-level semantic information, ultimately enabling simultaneous multi-states estimation. A filter-based multi-cues fusion object tracker is proposed for establishing more stable object-level data association. Combined with the object’s motion priors, the motion-aided feature tracking algorithm is proposed to improve the feature-level data association performance. Furthermore, a novel state estimation factor graph is designed which integrates a specific feature observation uncertainty model and the intrinsic priors of tracked object, and solved through sliding-window optimization. Our system is evaluated using the KITTI dataset and achieves comparable performance to state-of-the-art object pose estimation systems both quantitatively and qualitatively. We have also validated our system on simulation environment and a real-world dataset to confirm the potential application value in different practical scenarios.

Point cloud-based place recognition can be used for global localization in large-scale scenes and loop-closure detection in simultaneous localization and mapping (SLAM) systems in the absence of GPS. Current learning-based approaches aim to extract global and local features from 3D point clouds to encode them as descriptors for point cloud retrieval. The key problems are that the occlusion of point clouds by dynamic objects in the scene affects the point cloud structure, a single perceptual field of the network cannot adequately extract point cloud features, and the correlation between features is not fully utilized. To overcome this, we propose a novel network called CapsLoc3D. We first use the static point cloud generation module to remove the occlusion effects of dynamic objects, and then obtain the point cloud descriptors by processing with the CapsLoc3D network which contains the point spatial transformation module, multi-scale feature fusion module, Capsnet module and a GeM Pooling layer. After validation using the Oxford RobotCar, KITTI, and NEU datasets, experiments show that our method performs better and also has good generalization performance and computational efficiency compared with current state-of-the-art algorithms.

Due to seasonal and illumination variance, long-term visual localization tasks in dynamic environments is a crucial problem in the field of autonomous driving and robotics. At present, image-based retrieval is an effective method to solve this problem. However, it is difficult to completely distinguish changes in the same location over times by relying on content information alone. In order to solve these above problems, a double-domain network model combining semantic information and content information is proposed for visual localization task. In addition, this approach only needs to use the virtual KITTI 2 dataset for training. To reduce the domain difference between real scene and virtual image, the cross-predictive semantic segmentation mechanism is introduced to solve this problem. In addition, the obtained model achieves good domain adaptation and further has well generalization on other real datasets by introducing a domain loss function and a triplet semantic loss function. A series of experiments on the Extended CMU-Seasons dataset and the Oxford RobotCar-Seasons dataset demonstrates that the proposed network model outperformes the state-of-the-art baselines for retrieval-based visual localization in challenging environments.

Salient object detection (SOD) has rapidly developed in recent years, and detection performance has greatly improved. However, the price of these improvements is increasingly complex networks that require more computing resources and sacrifice real-time performance. This makes it difficult to deploy these approaches on devices with limited computing resources (such as mobile phones, embedded platforms, etc.). Considering recently developed lightweight SOD models, their detection and real-time performance are always compromised in demanding practical application scenarios. To solve these problems, we propose a novel lightweight SOD method called LARNet and its corresponding extremely lightweight method LARNet$^{*}$according to application requirements. These methods balance the relationship between lightweight requirements, detection accuracy and real-time performance. First, we propose a saliency backbone network tailored for SOD, which removes the need for pre-training with ImageNet and effectively reduces feature redundancy. Subsequently, we propose a novel context gating module (CGM), which simulates the physiological mechanism of human brain neurons and visual information processing, and realizes the deep fusion of multi-level features at the global level. Finally, the saliency map is output after fusion of multi-level features. Extensive experiments on popular benchmark datasets demonstrate that the proposed LARNet (LARNet$^{*}$) achieves 98 (113) FPS on a GPU and 3 (6) FPS on a CPU. With approximately 680 K (90 K) parameters, the model has significant performance advantages over (extremely) lightweight methods, even surpassing some heavyweight models.

Micro-expressions have been shown to be effective in understanding the genuine emotions of a person. While many advances have been made in detecting micro-expressions using deep learning, previous studies in recognizing micro-expressions require pre-processing steps and the use of large feature sets resulting in large runtimes and thus have limited applicability in real-world scenarios. In this paper, we propose time-efficient end-to-end framework which uses landmark-based positional features to generate spatio-temporal graphs that can be applied to micro-expression recognition using Graph Convolutional Neural Networks (GCNs). We explore the importance of landmark features and propose a selective feature reduction approach to further improve efficiency. We perform experiments using the SMIC, CASMEII and SAMM datasets and demonstrate that our approach significantly speeds up predictions and delivers results comparable to the state-of-the-art.

Real-time and robust long-term visual localization is a crucial technology for autonomous driving. Season and illumination variance make this problem more challenging. At present, most of excellent visual localization algorithms cannot run in real-time on devices with limited computing resources. In this paper, we propose SAMLoc, a structure-aware and self-supervised visual localization system, for fast and robust 6-DoF localization. To obtain structural features in the scene, we propose local and global structure-aware constraints using edge information. Then, we integrate the structure-aware constraints into the hierarchical localization network of multi-task distillation, which significantly reduces the feature extraction time while ensuring localization accuracy. As a result, real-time and robust large-scale localization can be achieved on mobile devices. Experimental results on public datasets show that our system can achieve high localization accuracy and have satisfactory real-time performance. Compared with several state-of-the-art visual localization systems, our framework achieves a competitive localization performance.

The progress of LiDAR-based 3D object detection has significantly enhanced developments in autonomous driving and robotics. However, due to the limitations of LiDAR sensors, object shapes suffer from deterioration in occluded and distant areas, which creates a fundamental challenge to 3D perception. Existing methods estimate specific 3D shapes and achieve remarkable performance. However, these methods rely on extensive computation and memory, causing imbalances between accuracy and real-time performance. To tackle this challenge, we propose a novel LiDAR-based 3D object detection model named BSH-Det3D, which applies an effective way to enhance spatial features by estimating complete shapes from a bird's eye view (BEV). Specifically, we design the Pillar-based Shape Completion (PSC) module to predict the probability of occupancy whether a pillar contains object shapes. The PSC module generates a BEV shape heatmap for each scene. After integrating with heatmaps, BSH-Det3D can provide additional information in shape deterioration areas and generate high-quality 3D proposals. We also design an attention-based densification fusion module (ADF) to adaptively associate the sparse features with heatmaps and raw points. The ADF module integrates the advantages of points and shapes knowledge with negligible overheads. Extensive experiments on the KITTI benchmark achieve state-of-the-art (SOTA) performance in terms of accuracy and speed, demonstrating the efficiency and flexibility of BSH-Det3D. The source code is available on https://github.com/mystorm16/BSH-Det3D.

This paper presents a learning based motion planning method for robotic manipulation, aiming to solve the asymptotically-optimal motion planning problem with nonlinear kinematics in a complex environment. The core of the proposed method is based on a novel neural network model, i.e., graph wasserstein autoencoder (GraphWAE) network, which is used to represent the implicit sampling distributions of the configuration space (C-space) for sampling-based planning algorithms. Through learning the implicit distributions, we can guide the planning process to search or extend in the desired region to reduce the collision checks dramatically for fast and high-quality motion planning. The theoretical analysis and proofs are given to demonstrate the probabilistic completeness and asymptotic optimality of the proposed method. Numerical simulations and experiments are conducted to validate the effectiveness of the proposed method through a series of planning problems from 2D, 6D and 12D robot C-spaces in the challenging scenes. Results indicate that the proposed method can achieve better planning performance than the state-of-the-art planning algorithms. Note to Practitioners—The motivation of this work is to develop a fast and high-quality asymptotically optimal motion planning method for practical applications such as autonomous driving, robotic manipulation and others. Due to the time consumption caused by collision detection, current planning algorithms usually take much time to converge to the optimal motion path especially in the complicated environment. In this paper, we present a neural network model based on GraphWAE to learn the biasing sampling distributions as the sample generation source to further reduce or avoid collision checks of sampling-based planning algorithms. The proposed method is general and can be also deployed in other sampling-based planning algorithms for improving planning performance in different robot applications.

Existing lightweight salient object detection (SOD) methods aim to solve the problem of high computational costs that is prevalent with heavyweight methods. However, compared with heavyweight methods, the detection accuracy of lightweight methods is greatly reduced while real-time performance is not significantly improved. Therefore, we aim to establish a trade off between computational cost and detection performance by improving the network efficiency. We propose a fast and extremely lightweight end-to-end wavelet neural network (ELWNet) for real-time salient object detection. ELWNet can achieve salient object detection and segmentation at approximately 70FPS (GPU), 19FPS (CPU) with 76K parameters and 0.38G FLOPs. We introduce wavelet transform theory into a neural network, proposing a wavelet transform module (WTM), a wavelet transform fusion module (WTFM), a novel feature residual mechanism, and construct an efficient architecture. The wavelet transform theory is integrated into the neural network to realize the interaction between the features in the frequency and the time domain. Meanwhile, ELWNet does not rely on a pre-trained model, which significantly reduces redundant features. We validate the performance of ELWNet using five well-known datasets, and demonstrate state-of-the-art performance compared with 24 other SOD models in terms of being lightweight, detection accuracy and real-time capabilities. Our method maintains high detection performance while reducing the number of model parameters by approximately 99% compared with heavyweight methods.

Industrial defect segmentation is important to ensure product quality and production safety. The main challenges in industrial applications are insufficient defect samples, large intraclass variation, and the interference of background information. However, most current texture defect segmentation methods rely on large-scale datasets and can only deal with one specific type of texture defect, which reduces the application efficiency and application scope of defect segmentation algorithms. To this end, we propose an optimal bilateral feature transport network (OBFTNet) for few-shot texture defect segmentation, which can accurately segment texture defects in multiple unseen materials (domains), such as steel, wood, and leather. OBFTNet can perform bilateral prediction for background and defect regions of unseen material by dynamically predicting task-specific semantic correspondences conditioned on a small guidance set. Specifically, we introduce background images (defect-free images) as supplementary learning information for reverse prediction and model the semantic correspondence between the guidance (support and background images) and the query images in few-shot segmentation as an optimal bilateral feature transport problem and generate a set of optimal bilateral correlation tensors. Using 4-D and 2-D convolutions, the model gradually reduces optimal bilateral correlation tensors to precise segmentation masks. Experimental results show that our proposed method outperforms several state-of-the-art techniques with very few labeled samples and the method generalizes well to industrial defects on unseen materials.

Object SLAM is a popular approach for autonomous driving and robotics, but accurate object perception in outdoor environments remains a challenge. State-of-the-art object SLAM algorithms rely on assumptions and are sensitive to observation noise, limiting their application in real-world scenarios. To address these challenges, we propose a novel object SLAM system that utilizes a quadric initialization algorithm based on constrained quadric optimization, which does not rely on planar assumptions and is robust to partial observations. Additionally, we introduce an automatic object data association algorithm capable of detecting motion states while associating objects across frames. To further enhance the accuracy of the quadric mapping, an extra thread is used to refine the ellipsoid parameters within a local sliding window composed of keyframes. Our system utilizes a joint optimization framework that optimizes camera poses, object landmarks, and point clouds in the local mapping thread for further global optimization while maintaining a consistent map. Experimental results on the real-world KITTI dataset show that the proposed system is more robust and significantly outperforms current state-of-the-art methods in quadric initialization and mapping in outdoor scenarios. Moreover, our system achieves real-time performance, making it suitable for practical applications.

Long-term visual place recognition (VPR) is challenging as the environment is subject to drastic appearance changes across different temporal resolutions, such as time of the day, month, and season. A wide variety of existing methods address the problem by means of feature disentangling or image style transfer but ignore the structural information that often remains stable even under environmental condition changes. To overcome this limitation, this article presents a novel structure-aware feature disentanglement network (SFDNet) based on knowledge transfer and adversarial learning. Explicitly, probabilistic knowledge transfer (PKT) is employed to transfer knowledge obtained from the Canny edge detector to the structure encoder. An appearance teacher module is then designed to ensure that the learning of appearance encoder does not only rely on metric learning. The generated content features with structural information are used to measure the similarity of images. We finally evaluate the proposed approach and compare it to state-of-the-art place recognition methods using six datasets with extreme environmental changes. Experimental results demonstrate the effectiveness and improvements achieved using the proposed framework. Source code and some trained models will be available at http://www.tianshu.org.cn.

Unsupervised person re-identification (re-ID) attracts growing attention due to its broad prospects in practical applications. State-of-the-art unsupervised re-ID approaches combine clustering-based pseudo-label prediction with feature fine-tuning. However, pseudo labels generated directly by clustering are not always reliable and inevitably contain noisy labels. To tackle these challenges, we propose a novel noise inhibition framework to estimate the confidence of each pseudo label and actively correct noisy labels. By introducing the silhouette coefficient, our method can estimate the pseudo-label confidence without any extra model or data, and calculate the correction matrix to correct clustering results directly. However, the silhouette coefficient is usually applied on the hyper-parameters selection of clustering algorithms. In order to make the silhouette coefficient more suitable for estimation and correction tasks, we calibrate the Jaccard distance matrix to alleviate the negative influence of the cluster size on the silhouette coefficient. Our proposed method brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.

Deep learning based salient object detection methods have recently received significant attention. However, current methods still suffer from shortcomings such as informative background information being ignored which a significant problem for image saliency understanding. Additionally, it is also a challenge to suppress the noisy features in the network. By analyzing the difference between high-level and low-level features from ResNet-50, we utilize a Bilateral Feature Fusion (BFF) module to deal with the problem caused by ignoring informative background information. Benefitting from the BFF module, our proposed network can capture more meaningful foreground and background cues, which helps to get a more accurate saliency map. Moreover, we adopt a Score Sorting Attention (SSA) module which suppresses noisy and irrelevant features. Experimental results on five benchmark datasets demonstrate that our proposed method performs better than other state-of-the-art methods. The ablation studies also prove our contributions.

Automated inspection has become a vital part of quality control during semiconductor wafer production. Current processes are focused on finding defects via variation from a ‘golden' image using pixel to pixel comparisons or utilization of opaque neural network-based approaches. In this paper we present an approach which uses deep learning methods to classify defects on semiconductor die images and show the experimental steps taken in order to produce a highly accurate system based on previous models.

Blind face restoration (BFR) from severely degraded face images is important in face image processing, and has attracted increasing attention due to its wide applications. How-ever, due to the complex unknown degradations in real-world scenarios, existing priors-based methods tend to restore faces with unstable quality. In this paper, we propose a Facial Landmarks and Generative Priors Guided Blind Face Restoration Network (FGPNet) to seamlessly integrate the advantages of generative priors and face-specific geometry priors. Specifically, we pretrain a high-quality (HQ) face synthesis generative adversarial network (GAN) and a landmarks prediction network, and then embed them into a U-shaped deep neural network (DNN) as decoder priors to guide face restoration, during which the generative priors can provide adequate details and the landmarks priors provide geometry and semantic information. Furthermore, we design facial priors fusion (FPF) blocks to incorporate the prior features from pretrained face synthesis GAN and landmarks prediction network in an adaptive and progressive manner, making our FGPNet exhibits good generalization in real-world application. Experiments demonstrate the superiority of our FGPNet in comparison to state-of-the-arts, and also show its potential in handling real-world low-quality images from several practical applications.

Loop closure detection has the potential to correct the drift of trajectories and build a global consistent map in LiDAR SLAM, however it remains a challenging problem in outdoor environment due to the sparsity of 3D point clouds data, large-scale scenes and moving objects. Inspired by the way humans perceive the environment through recognizing objects and identifying their relations, this paper presents a novel descriptor that contains semantic and topological information for loop closure detection. Unlike most existing methods that extract features from the raw point clouds or use all semantic objects, we directly discard point clouds representing pedestrians and vehicles after semantic segmentation. Then, we propose a semantic topological graph representation from the remaining point clouds and convert this graph into a descriptor. Additionally, we propose a two-stage algorithm for matching descriptors to efficiently determine the loop. Our method has been extensively evaluated using the KITTI dataset and outperforms state-of-the-art methods, especially in the challenging situations such as viewpoint changes and dynamic scenes.

Person re-identification (ReID) is a crucial aspect of recognising pedestrians across multiple surveillance cameras. Even though significant progress has been made in recent years, the viewpoint change and scale variations still affect model performance. In this paper, we observe that it is beneficial for the model to handle the above issues when boost the consistent feature extraction capability among different transforms (e.g., flipping and scaling) of the same image. To this end, we propose a visual attention consistency network (VAC-Net). Specifically, we propose Embedding Spatial Consistency (ESC) architecture with flipping, scaling and original forms of the same image as inputs to learn a consistent embedding space. Furthermore, we design an Input-Wise visual attention consistent loss (IW-loss) so that the class activation maps(CAMs) from the three transforms are aligned with each other to enforce their advanced semantic information remains consistent. Finally, we propose a Layer-Wise visual attention consistent loss (LW-loss) to further enforce the semantic information among different stages to be consistent with the CAMs within each branch. These two losses can effectively improve the model to address the viewpoint and scale variations. Experiments on the challenging Market-1501, DukeMTMC-reID, and MSMT17 datasets demonstrate the effectiveness of the proposed VAC-Net.

Most existing attention mechanisms have no supervised signal during the training phase, which limits the model feature learning capability. To solve this problem, we propose a novel parameter-free attention mechanism based on class activation mapping. Attention mechanisms usually consist of spatial attention and channel attention, which indicates that “where” and “what” is more meaningful, respectively. Our attention also contains both types of attention. For Spatial Attention, we use class activation mapping as a supervision signal to guide the generation of it directly in space. Thus our spatial attention can pay more attention to the informative pedestrian parts of the scene and reduce background interference. For Channel Attention, the importance of each channel is obtained by the similarity between the aforementioned spatial attention and the feature map of each channel. In this manner, our channel attention is indirectly guided by class activation mapping. In addition, our attention is parameter-free, which reduces the risk of over-fitting. Finally, we conduct extensive evaluations on three popular benchmark datasets including Market1501, DukeMTMC-reID, and MSMT17, demonstrating the effectiveness of our approach on discriminative person representations.

In this paper, a data assimilation network is proposed to tackle the challenges of domain generalization for person re-identification (ReID). Most of the existing research efforts only focus on single-dataset issues, and the trained models are difficult to generalize to unseen scenarios. This paper presents a distinctive idea to improve the generality of the model by assimilating three types of images: style-variant images, misaligned images and unlabeled images. The latter two are often ignored in the previous domain generalization ReID studies. In this paper, a non-local convolutional block attention module is designed for assimilating the misaligned images, and an attention adversary network is introduced to correct it. A progressive augmented memory is designed for assimilating the unlabeled images by progressive learning. Moreover, we propose an attention adversary difference loss for attention correction, and a labeling-guide discriminative embedding loss for progressive learning. Rather than designing a specific feature extractor that is robust to style shift as in most previous domain generalization work, we propose a data assimilation meta-learning procedure to train the proposed network, so that it learns to assimilate style-variant images. It is worth mentioning that we add an unlabeled augmented dataset to the source domain to tackle the domain generalization ReID tasks. Extensive experiments demonstrate that our approach significantly outperforms the state-of-the-art domain generalization methods.

This paper presents the first active object mapping framework for complex robotic manipulation and autonomous perception tasks. The framework is built on an object SLAM system integrated with a simultaneous multi-object pose estimation process that is optimized for robotic grasping. Aiming to reduce the observation uncertainty on target objects and increase their pose estimation accuracy, we also design an object-driven exploration strategy to guide the object mapping process, enabling autonomous mapping and high-level perception. Combining the mapping module and the exploration strategy, an accurate object map that is compatible with robotic grasping can be generated. Additionally, quantitative evaluations also indicate that the proposed framework has a very high mapping accuracy. Experiments with manipulation (including object grasping and placement) and augmented reality significantly demonstrate the effectiveness and advantages of our proposed framework.

Scale ambiguity is a fundamental problem in monocular visual odometry. Typical solutions include loop closure detection and environment information mining. For applications like self-driving cars, loop closure is not always available, hence mining prior knowledge from the environment becomes a more promising approach. In this paper, with the assumption of a constant height of the camera above the ground, we develop a light-weight scale recovery framework leveraging an accurate and robust estimation of the ground plane. The framework includes a ground point extraction algorithm for selecting high-quality points on the ground plane, and a ground point aggregation algorithm for joining the extracted ground points in a local sliding window. Based on the aggregated data, the scale is finally recovered by solving a least-squares problem using a RANSAC-based optimizer. Sufficient data and robust optimizer enable a highly accurate scale recovery. Experiments on the KITTI dataset show that the proposed framework can achieve state-of-the-art accuracy in terms of translation errors, while maintaining competitive performance on the rotation error. Due to the light-weight design, our framework also demonstrates a high frequency of 20 Hz on the dataset.

The ability to quickly and accurately triage a person’s medical condition in an emergency situation or other critical scenarios could mean the difference between life and death. Endowing a robotic system with vision and tactile capabilities, similar to those of medical professionals, and thus enabling robots to assess a patient’s status in an emergency is a highly sought after characteristic in healthcare robotics. This paper presents a novel fuzzy triage system exploiting visual and tactile sensing, to equip a robot with the skills to accurately determine key vital signs in humans. There are three key signs of human health: respiratory rate, pulse rate (Beats Per Minute (BPM)) and capillary refill time. Using ground truth from a medical professional, the fuzzy triage system is trained and validated initially with informed synthetic data and then further evaluated using vital signs data collected from subjects in a pilot study. Results from this pilot study indicate that the fuzzy triage system is capable of classifying a patient’s health using the novel approaches for collecting BPM, Respiratory Rate (RR) and Capillary Refill Time (CRT) which replicate, to some extent, the approaches used by medical professionals for measuring vital signs. Furthermore, the intelligent system proved capable of determining whether a pulse was regular or arrhythmic, whether respiratory rate was regular or irregular, and determining the subject’s capillary refill time. Such results imply that this system could ultimately be used, for example, in a home assistance robot for elderly or disabled persons, or as a first responder robot. Ultimately the aim would be that these methods could be utilised by robotic systems in emergency scenarios or disaster zones.

Neural coding schemes are powerful tools used within neuroscience. This paper introduces three different neural coding scheme formations for event-based vision data which are designed to emulate the neural behaviour exhibited by neurons under stimuli. Presented are phase-of-firing and two sparse neural coding schemes. It is determined that machine learning approaches, i.e. Convolutional Neural Network combined with a Stacked Autoencoder network, produce powerful descriptors of the patterns within events. These coding schemes are deployed in an existing action recognition template and evaluated using two popular event-based data sets.

Event-based image processing is a relatively new domain in the field of computer vision. Much research has been carried out on adapting event-based data to comply with established techniques from frame-based computer vision. On the contrary, this paper presents a descriptor which is designed specifically for direct use with event-based data and therefore can be considered to be a pure event-based vision descriptor as it only uses events emitted from event-based vision devices without transforming the data to accommodate frame-based vision techniques. This novel descriptor is known as the Post-stimulus Time-dependent Event Descriptor (P-TED). P-TED is comprised of two features extracted from event data which describe motion and the underlying pattern of transmission respectively. Furthermore a framework is presented which leverages the P-TED descriptor to classify motions within event data. This framework is compared against another state-of-the-art event-based vision descriptor as well as an established frame-based approach.

Machine learning usage for forecasting is popular in financial trading, particularly for stock price prediction and this is often combined with technical indicators to extract key predictive indicators from large time series trading datasets. Energy market trading data have similar characteristics to financial trading data, therefore deriving technical indicators specifically for electricity prices will help predict future prices and reduce trading costs. We have derived eight technical indicators for the Integrated Single Electricity Market (ISEM) energy market in Ireland using hourly electricity price data over the period February 2019 until November 2019. Technical indicator based models were obtained by using machine learning regression algorithms (Extreme Gradient [XG] Boost, Random Forest, and Gradient Boosting) trained with the proposed novel technical indicators. The results of the technical indicator models were compared against the baseline model (raw price data only) to see if using technical indicators as inputs improves model performance. We conclude that electricity prices can be accurately predicted using the proposed technical indicators.

This paper presents a novel Reducing-Over-Time (ROT) binary tree structure for event-based vision data and subtypes of the tree structure. A framework is presented using ROT, that takes advantage of the self-balancing and self-pruning nature of the tree structure to extract spatial-temporal information. The ROT framework is paired with an established motion classification technique and performance is evaluated against other state-of-the-art techniques using four datasets. Additionally, the ROT framework as a processing platform is compared with other event-based vision processing platforms in terms of memory usage and is found to be one of the most memory efficient platforms available.

Building an effective automatic micro expression recognition (MER) system is becoming increasingly desirable in computer vision applications. However, it is also very challenging given the fine-grained nature of the expressions to be recognized. Hence, we investigate if amplifying micro facial muscle movements as a pre-processing phase, by employing Eulerian Video Magnification (EVM), can boost performance of Local Phase Quantization with Three Orthogonal Planes (LPQ-TOP) to achieve improved facial MER across various datasets. In addition, we examine the rate of increase for recognition to determine if it is uniform across datasets using EVM. Ultimately, we classify the extracted features using Support Vector Machines (SVM). We evaluate and compare the performance with various methods on seven different datasets namely CASME, CAS(ME)2, CASME2, SMIC-HS, SMIC-VIS, SMIC-NIR and SAMM. The results obtained demonstrate that EVM can enhance LPQ-TOP to achieve improved recognition accuracy on the majority of the datasets.

In recent years, brain-controlled intelligent wheelchairs have received extensive attention, which combines the accessibility of the Brain-computer Interface (BCI) system with the intelligence of wheelchairs. However, current brain-controlled wheelchairs are always operated in a fixed mode. The Electroencephalogram (EEG) signals with the fixed acquisition time are analyzed without considering the state of the user, which not only increases the risk of misoperation, but seriously reduces the information transfer rate of the system. To solve this problem, an adaptive control approach for intelligent wheelchair based on BCI combining with Quality of Operating (QoO) is proposed. Firstly, the influence of motor imagery signals with different time lengths in different states on classification accuracy was analyzed using tangent space Support Vector Machine (TSSVM) algorithm. Then, the definition of QoO was introduced, which was obtained by analyzing sample entropy and power spectral density (PSD) of four kinds of EEG activities, delta, theta, alpha and beta. Finally, the acquisition time of required EEG signals was adjusted according to the value of QoO. We constructed a brain-controlled wheelchair system and conducted real environmental experiments for 9 subjects using strategies, with and without adaptive control approach. The results show that the approach proposed in this paper can reduce the risk of misoperation and increase the information transfer rate on the premise of ensuring the classification performance during navigation in complex indoor environment.

Activity recognition relates to the automatic visual detection and interpretation of human behaviour and is emerging as an active domain of computer vision. It has important applications such as identifying individuals who are at risk of suicide in public locations such as bridges or railway stations. These individuals are known to exhibit easily observable activities and behaviours such as pacing, looking up and down the railway tracks, and leaving objects on the platform. In order to detect these behaviours, an approach to individual person activity recognition is needed which can run in real time and monitor multiple individuals in parallel. We present a method for human activity recognition using skeletal keypoints and investigate how using varying sample rates and sequence lengths impacts accuracy. The results show that for any given sequence length, optimising the sample rate can result in an overall increase in classification accuracy and improvement in run-time. Results demonstrate that finding the optimal time period over which to sample frames is more important than simply decreasing the number of frames sampled. Further, we show that keypoint based activity recognition approaches outperform other state of the art approaches. Finally, we show that this approach is fast enough for real time activity recognition when up to 14 people are present in the image whilst maintaining a high degree of accuracy.

Object-level data association and pose estimation play a fundamental role in semantic SLAM, which remain unsolved due to the lack of robust and accurate algorithms. In this work, we propose an ensemble data associate strategy for integrating the parametric and nonparametric statistic tests. By exploiting the nature of different statistics, our method can effectively aggregate the information of different measurements, and thus significantly improve the robustness and accuracy of data association. We then present an accurate object pose estimation framework, in which an outliers-robust centroid and scale estimation algorithm and an object pose initialization algorithm are developed to help improve the optimality of pose estimation results. Furthermore, we build a SLAM system that can generate semi-dense or lightweight object-oriented maps with a monocular camera. Extensive experiments are conducted on three publicly available datasets and a real scenario. The results show that our approach significantly outperforms state-of-the-art techniques in accuracy and robustness. The source code is available on https://github.com/yanmin-wu/EAO-SLAM.

The generalization ability of deep person re-identification networks is subject to inadequate person data and occlusions. To relieve this dilemma, we propose a feature-level augmentation strategy, Adversarially Erased Learning Module (AELM), using two adversarial classifiers. Specifically, we utilize a classifier to identify discriminative regions and erase them to increase the variant of features. Meanwhile, we input the erased feature maps to another classifier to discover new body regions, which effectively resist occlusion of key parts. To easily perform end-to-end training for AELM, we propose a novel Identity model based on Fully Convolutional Networks (IFCN) to directly obtain body response heatmap during the forward pass by selecting corresponding class-specific feature map. Thus, the discriminative regions can be identified and erased in a convenient way. Moreover, to capture discriminative region for AELM, we present a Complementary Attention Module (CoAM) combined with channel and spatial attention to automatically focus on which feature types and positions are meaningful in the feature maps. In this paper, CoAM and AELM are cascaded into one module which is applied to the outputs of different convolutional layers to integrate mid- and high-level semantic features. Experimental results on three challenging benchmarks demonstrate the effectiveness of the proposed method.

Inflammatory Bowel Disease (IBD) is an umbrella term for a group of inflammatory diseases of the gastrointestinal tract, including Crohn's Disease and ulcerative colitis. Changes to the intestinal microbiome, the community of micro-organisms that resides in the human gut, have been shown to contribute to the pathogenesis of IBD. IBD diagnosis is often delayed due to its non-specific symptoms and because an invasive colonoscopy is required for confirmation, which leads to poor growth in children and worse treatment outcomes. Feature selection algorithms are often applied to microbial communities to identify bacterial groups that drive disease. It has been shown that aggregating Ensemble Feature Selection (EFS) can improve the robustness of feature selection algorithms, which is defined as the variation of feature selector output caused by small changes to the dataset. In this work, we apply a two-step filter and an EFS process to generate robust feature subsets that can non-invasively predict IBD subtypes from high-resolution microbiome data. The predictive power of the robust feature subsets is the highest reported in literature to date. Furthermore, we identify five biologically plausible bacterial species that have not previously been implicated in IBD aetiology.

With the ever-increasing global population and average life expectancy, care homes and care at home services are continuously being stretched beyond capacity. Recent developments in tactile sensing have enabled robot systems to measure human vital signs such as beats per minute (BPM), Respiratory Rate (RR) and Capillary Refill Time (CRT). Using robotic systems to measure vital sign data in the home of an elderly or disabled person would greatly assist medical and health services. This paper proposes the use of a vital sign measuring robotic system together with Cloud computing to intelligently process big data and ascertain the current health status of the service user without the need to expose their identity or burden health professionals. Furthermore, a method that enables medical professionals to visualise the data for a complete geographical region as well as for individual patients is presented and hence we provide details of a closed loop system to support ageing-in-place.

Determining the receptive field of a retinal ganglion cell is critically important when formulating a computational model that maps the relationship between the stimulus and response. This process is traditionally undertaken using reverse correlation to estimate the receptive field. By stimulating the retina with artificial stimuli, such as alternating checkerboards, bars or gratings and recording the neural response it is possible to estimate the cell’s receptive field by analysing the stimuli that produced the response. Artificial stimuli such as white noise is known to not stimulate the full range of the cell’s responses. By using natural image stimuli, it is possible to estimate the receptive field and obtain a resulting model that more accurately mimics the cells’ responses to natural stimuli. This paper extends on previous work to seek further improvements in estimating a ganglion cell’s receptive field by considering that the receptive field can be divided into subunits. It is thought that these subunits may relate to receptive fields which are associated with bipolar retinal cells. The findings of this preliminary study show that by using subunits to define the receptive field we achieve a significant improvement over existing approaches when deriving computational models of the cell’s response.

In recent years, artificial vision research has moved from focusing on the use of only intensity images to include using depth images, or RGB-D combinations due to the recent development of low-cost depth cameras. However, depth images require a lot of storage and processing requirements. In addition, it is challenging to extract relevant features from depth images in real time. Researchers have sought inspiration from biology in order to overcome these challenges resulting in biologically inspired feature extraction methods. By taking inspiration from nature, it may be possible to reduce redundancy, extract relevant features, and process an image efficiently by emulating biological visual processes. In this paper, we present a depth and intensity image feature extraction approach that has been inspired by biological vision systems. Through the use of biologically inspired spiking neural networks, we emulate functional computational aspects of biological visual systems. The results demonstrate that the proposed bioinspired artificial vision system has increased performance over existing computer vision feature extraction approaches.

The processing capabilities of biological vision systems are still vastly superior to artificial vision, even though this has been an active area of research for over half a century. Current artificial vision techniques integrate many insights from biology yet they remain far-off the capabilities of animals and humans in terms of speed, power, and performance. A key aspect to modeling the human visual system is the ability to accurately model the behavior and computation within the retina. In particular, we focus on modeling the retinal ganglion cells (RGCs) as they convey the accumulated data of real world images as action potentials onto the visual cortex via the optic nerve. Computational models that approximate the processing that occurs within RGCs can be derived by quantitatively fitting the sets of physiological data using an input-output analysis where the input is a known stimulus and the output is neuronal recordings. Currently, these input-output responses are modeled using computational combinations of linear and nonlinear models that are generally complex and lack any relevance to the underlying biophysics. In this paper, we illustrate how system identification techniques, which take inspiration from biological systems, can accurately model retinal ganglion cell behavior, and are a viable alternative to traditional linear-nonlinear approaches.

Currently, hexagonal image processing is mainly based on simulated data generated by square lattice conversions. For the evaluation of the conversion quality, this paper presents a method for ideal square to hexagonal lattice conversion. Based on the square lattice discrete-time Fourier transform (DTFT), the method first determines the values of the hexagonal discrete Fourier transform (HDFT), and performs the inverse HDFT to obtain the ideal conversion. This method provides a benchmark for evaluating other practical conversions and such evaluation is presented in this paper.

A fundamental problem in cooperative HumanRobot Interaction is object handover. Existing works in this area assume the human can reliably grasp the object from the robot hand. However, in some situations the human can produce perturbing forces in the object that are not meant to end in a handover. These perturbations can result in the object being dropped or the robot hand being damaged. This paper addresses this problem and presents a mechanism for reliable robot to human object handover implemented in a Shadow Robot hand endowed with tactile sensing. Given a stable grasping configuration, using BioTAC sensors we are able to estimate the contact forces applied to the object, and provide a feedback signal to a joint effort controller to maintain grasp forces despite perturbations. Our system is able to identify between object pulling forces which should result in an object handover, and other disturbances. Experimental results show that the hand releases the object only when the object is pulled, validating the proposed algorithm.

A fundamental aspect of many human-robot collaborative tasks is object exchange or handover. Several techniques have been proposed to decide when a robot hand or gripper should release an object for a human to receive. However, these techniques typically neglect the reliability of the handover, assuming the process will occur without issue. Based on the fact that humans apply pulling forces in specific directions when receiving an object, this paper presents a recursive procedure enabling a robot to release an object appropriately and timely during a handover. Experiments with naive users showed subject-specific consistent pulling directions during robot-human handovers, highlighting the need for a system to be capable of detecting force directions in relation to objects. The approach reported in this paper shows that through tactile sensing the proposed approach can accurately classify between five different actions impacting an object held by a robot hand. The recursive nature of the system also enables detection of sequences of different actions, enabling the robot to decide to safely release the object only when the pulling performed by the human is in the right direction.

The creation of a predictive system that correctly forecasts future changes of a stock price is crucial for investment management and algorithmic trading. The use of technical analysis for financial forecasting has been successfully employed by many researchers. Input window length is a time frame parameter required to be set when calculating many technical indicators. This study explores how the performance of the predictive system depends on a combination of a forecast horizon and an input window length for forecasting variable horizons. Technical indicators are used as input features for machine learning algorithms to forecast future directions of stock price movements. The dataset consists of ten years daily price time series for fifty stocks. The highest prediction performance is observed when the input window length is approximately equal to the forecast horizon. This novel pattern is studied using multiple performance metrics: prediction accuracy, winning rate, return per trade and Sharpe ratio.

A system of interacting agents is, by definition, very demanding in terms of computational resources. Although multi-agent systems have been used to solve complex problems in many areas, it is usually very difficult to perform large-scale simulations in their targeted serial computing platforms. Reconfigurable hardware, in particular Field Programmable Gate Arrays devices, have been successfully used in High Performance Computing applications due to their inherent flexibility, data parallelism, and algorithm acceleration capabilities. Indeed, reconfigurable hardware seems to be the next logical step in the agency paradigm, but only a few attempts have been successful in implementing multi-agent systems in these platforms. This article discusses the problem of inter-agent communications in Field Programmable Gate Arrays. It proposes a Network-on-Chip in a hierarchical star topology to enable agents’ transactions through message broadcasting using the Open Core Protocol as an interface between hardware modules. A customizable router microarchitecture is described and a multi-agent system is created to simulate and analyse message exchanges in a generic heavy traffic load agent-based application. Experiments have shown a throughput of 1.6Gbps per port at 100MHz without packet loss and seamless scalability characteristics.

The emergence of social media has provided vast amounts of information that is potentially valuable for emergency management. In the EU-FP7 Project Security Systems for Language and Image Analysis (Slandail), an image analysis system has been developed to recognize the flood water images from the social media resources by incorporating with text analysis. A novel image feature descriptor has been developed to facilitate fast image processing based on incorporation of the "Squiral" (Square-Spiral) Image Processing (SIP) framework with the "Speeded-up Robust Features" (SURF). A new approach is proposed to generate an index from image recognition outcomes based on a moving window average, which presents a temporal change based on the occurrence of flooding water identified by image analysis. The evaluation for computation time and recognition were based on a batch of images obtained from the US Federal Emergency Management Agency (FEMA) media library and Facebook corpus from Germany, and the outcomes show the advantages of the proposed image features. The simulation results demonstrate the concept of the index based on a moving window average, highlighting the potential for application in emergency management.

Tactile sensing has recently attracted significant research interest in robotics. Despite the fact that tactile sensors provide temporal sequences of readings, state-of-the-art material recognition approaches are episodic, i.e. a whole sequence of readings is processed to identify the material. Based on vibration frequency response, this work presents an online identification technique using recursive estimation of the probability of identifying a set of materials, i.e. casting the classification problem as a hidden Markov model (HMM) state estimation problem. This allows for faster identification of most materials and does not require several exploratory movements. Our results show that when enough evidence is gathered, the system eventually achieves perfect recognition of our experimental set of 34 materials with an average identification time of ≤ 0.5 seconds. To prove the accuracy of this method, we also conducted a comparative experiment with commonly used machine learning algorithms for material identification such as k-Nearest Neighbour(KNN), an Artificial Neural Network(ANN) and Support Vector Machine(SVM).

Tinnitus is the phantom perception of a sound heard in or around the head in the absence of an identifiable source affecting 10-15% worldwide. The majority of tinnitus sufferers have some form of hearing loss. The multiple pathologies that generate and sustain tinnitus in a diverse tinnitus population make it challenging to establish a homogeneous cohort for experimental studies. People with no hearing loss or previous experience of tinnitus also begin to perceive phantom sounds when situated in a sound proof room for five minutes or less. This is consistently observed across multiple studies. Studies that induced tinnitus through acoustic deprivation in healthy subjects provide a more controlled environment to observe tinnitus. Although experimental work shows what is happening it does not explain how the tinnitus related activity is generated. Computational modelling of tinnitus following hearing loss has shown that underlying mechanisms, such as adaptive gain, can generate hyperactivity in the regions of hearing loss. These models do not account for the generation of tinnitus in people with no hearing loss. In this work we model the development of tinnitus related activity in cases of no hearing loss and induced acoustic deprivation. The tinnitus related activity disappears once the model is returned to normal ambient noise.

Inflammatory bowel disease (IBD) is a group of inflammatory diseases of the human colon and small intestine. IBD symptoms are non-specific; diagnosis can be delayed because an invasive colonoscopy is required for confirmation. Delayed diagnosis is linked to poor growth in children. Imbalances in the human intestinal microbiome - the community of microorganisms that reside in the human gut - are thought to contribute to the development of IBD. Work done to date in classifying host health statuses from patterns in human microbiomes with supervised learning algorithms has focused on modelling what is present in the gut (i.e. a bacterial census) with the random forest algorithm. Metagenomic shotgun sequencing is required to understand what is occurring in the gut (i.e. gene functions) and is often cost prohibitive for hundreds of samples. However, gene functions can be predicted with the Phylogenetic Investigation of Communities by Reconstruction of Unobserved States (PiCRUSt) software package, which could represent a valuable source of new features. In this paper we investigate feature relevance across the feature set with the Boruta algorithm. We find that the majority of relevant features are from the predicted metagenome. Support vector machines (SVM) and multilayer perceptrons (MLP) are rarely used with microbiomic datasets but offer several theoretical advantages. To determine if the new features and alternative algorithms are appropriate, we experiment with a range of machine learning and computational intelligence algorithms. With the best performing algorithms we also implement a conditional multiple classifier system that can identify IBD presence, IBD subtype, and IBD activity from a non-invasive stool sample.

Tactile sensing has been used in robotics for object identification, grasping, and material recognition. Most material recognition approaches use vibration signals from a tactile exploration, typically above one second long, to identify the material. This work proposes a tactile multi-modal (vibration and thermal) material identification approach based on recursive Bayesian estimation. Through the frequency response of the vibration induced by the material and thermal features, like an estimate of the thermal power loss of the finger, we show that it is possible to identify materials in less than half a second. Moreover, a comparison between vibration only and multi-modal identification shows that both recognition time and classification errors are reduced by adding thermal information.

In recent years, the processing of hexagonal pixel-based images has been investigated, and as a result, a number of edge detection algorithms for direct application to such image structures have been developed. We build on this paper by presenting a novel and efficient approach to the design of hexagonal image processing operators using linear basis and test functions within the finite element framework. Development of these scalable first order and Laplacian operators using this approach presents a framework both for obtaining large-scale neighborhood operators in an efficient manner and for obtaining edge maps at different scales by efficient reuse of the seven-point linear operator. We evaluate the accuracy of these proposed operators and compare the algorithmic performance using the efficient linear approach with conventional operator convolution for generating edge maps at different scale levels.

A wash trade refers to the illegal activities of traders who utilize carefully designed limit orders to manually increase the trading volumes for creating a false impression of an active market. As one of the primary formats of market abuse, a wash trade can be extremely damaging to the proper functioning and integrity of capital markets. The existing work focuses on collusive clique detections based on certain assumptions of trading behaviors. Effective approaches for analyzing and detecting wash trade in a real-life market have yet to be developed. This paper analyzes and conceptualizes the basic structures of the trading collusion in a wash trade by using a directed graph of traders. A novel method is then proposed to detect the potential wash trade activities involved in a financial instrument by first recognizing the suspiciously matched orders and then further identifying the collusions among the traders who submit such orders. Both steps are formulated as a simplified form of the knapsack problem, which can be solved by dynamic programming approaches. The proposed approach is evaluated on seven stock data sets from the NASDAQ and the London Stock Exchange. The experimental results show that the proposed approach can effectively detect all primary wash trade scenarios across the selected data sets.

Fast image processing is a key element in achieving real-time image and video analysis. The spiral addressing scheme [10] has been an efficient tool for hexagonal image processing (HIP), whereby the image pixel indices are stored in a one-dimensional vector that enables fast processing. Unlike HIP, which requires a complex resampling scheme, we present a novel “squiral” (square spiral) image processing (SIP) framework that provides a spiral addressing scheme for direct application to standard square pixel-based images. A SIP-based non-overlapping convolution technique is developed by simulating the eye tremor phenomenon of the human visual system to accelerate computation in feature extraction. Furthermore, we deploy the proposed simulated eye tremor technique on a sequence of video frames. The preliminary results based on two action video clips demonstrate the potential of the SIP-based eye tremor model to facilitate fast video processing.

Pulse rate and rhythm are indicators of the health of a human's blood circulation. Being able to detect one's pulse rate and rhythm in an emergency situation could be the difference between life and death. The work presented in this paper is preliminary work on algorithms that will equip a robot with the necessary skills to assess a human's pulse. Algorithms for pulse detection and the calculation of beats per minute (bpm) and Pulse to Pulse Interval (PPI) are presented and used to classify if a heart rate is normal, bradycardic, or tachycardic. Furthermore PPI is used to determine if the pulse rate is regular or in a form of arrhythmia. The results in this paper show that pulse was successfully detected and a subjects bpm calculated using a trough detection method. Furthermore the system proved to be capable of classifying the pulse as being regular or arrhythmic.

Even though artificial vision has been in development for over half a century it still fares poorly when compared to biological vision. The processing capabilities of biological visual systems are vastly superior in terms of power, speed, and performance. Inspired by this robust performance artificial vision systems have sought to take inspiration from biology by modeling aspects of biological vision systems. Existing computational models of visual neurons can be derived by quantitatively fitting particular sets of physiological data using an input-output analysis where a known input is given to the system and its output is recorded. These models need to capture the full spatio-temporal description of neuron behaviour under natural viewing conditions. In this work we use state-of-the-art fuzzy neural network techniques to accurately model the responses of retinal ganglion cells. We illustrate how a self-organising fuzzy neural network can accurately model ganglion cell behaviour, and are a viable alternative to traditional system identification techniques.

Accurate forecasting of upcoming trends in the capital markets is extremely important for algorithmic trading and investment management. Before making a trading decision, investors estimate the probability that a certain news item will influence the market based on the available information. Speculation among traders is often caused by the release of a breaking news article and results in price movements. Publications of news articles influence the market state that makes them a powerful source of data in financial forecasting. Recently, researchers have developed trend and price prediction models based on information extracted from news articles. However, to date no previous research that investigates the advantages of using news articles with different levels of relevance to the target stock has been conducted. This research study uses the multiple kernel learning technique to effectively combine information extracted from stock-specific and sub-industry-specific news articles for prediction of an upcoming price movement. News articles are divided into these two categories based on their relevance to a targeted stock and analyzed by separate kernels. The experimental results show that utilizing two categories of news improves the prediction accuracy in comparison with methods based on a single news category.

Modelling aspects of the human vision system, including the retina, is difficult due to insufficient knowledge about the internal components, organisation and complexity of the interactions within the system. Retinal ganglion cells are considered a core component of the human visual system as they convey the accumulated data as action potentials onto the optic nerve. Current techniques capable of mapping this input-output response involve computational combinations of linear and nonlinear models that are generally complex and lack any relevance to the underlying biophysics. This paper aims to model a retinal ganglion cell with a simple spiking neuron combined with a pre-processing method, which accounts for the preceding retinal neural structure. Performance of the models is compared with the spike responses obtained in the electrophysiological recordings from a mammalian retina subjected to visual stimulation.

Despite the emerging applications of diffusion tensor imaging (DTI) to mild traumatic brain injury (mTBI), very few investigations have been reported related to temporal changes in quantitative diffusion patterns, which may help to assess recovery from head injury and the long term impact associated with cognitive and behavioral impairments caused by mTBI. Most existing methods are focused on detection of mTBI affected regions rather than quantification of temporal changes following head injury. Furthermore, most methods rely on large data samples as required for statistical analysis and, thus, are less suitable for individual case studies. In this paper, we introduce an approach based on spatial group independent component analysis (GICA), in which the diffusion scalar maps from an individual mTBI subject and the average of a group of controls are arranged according to their data collection time points. In addition, we propose a constrained GICA (CGICA) model by introducing the prior information into the GICA decomposition process, thus taking available knowledge of mTBI into account. The proposed method is evaluated based on DTI data collected from American football players including eight controls and three mTBI subjects (at three time points post injury). The results show that common spatial patterns within the diffusion maps were extracted as spatially independent components (ICs) by GICA. The temporal change of diffusion patterns during recovery is revealed by the time course of the selected IC. The results also demonstrate that the temporal change can be further influenced by incorporating the prior knowledge of mTBI (if available) based on the proposed CGICA model. Although a small sample of mTBI subjects is studied, as a proof of concept, the preliminary results provide promising insight for applications of DTI to study recovery from mTBI and may have potential for individual case studies in practice.

Price manipulation refers to the activities of those traders who use carefully designed trading behaviors to manually push up or down the underlying equity prices for making profits. With increasing volumes and frequency of trading, price manipulation can be extremely damaging to the proper functioning and integrity of capital markets. The existing literature focuses on either empirical studies of market abuse cases or analysis of particular manipulation types based on certain assumptions. Effective approaches for analyzing and detecting price manipulation in real time are yet to be developed. This paper proposes a novel approach, called adaptive hidden Markov model with anomaly states (AHMMAS) for modeling and detecting price manipulation activities. Together with wavelet transformations and gradients as the feature extraction methods, the AHMMAS model caters to price manipulation detection and basic manipulation type recognition. The evaluation experiments conducted on seven stock tick data from NASDAQ and the London Stock Exchange and 10 simulated stock prices by stochastic differential equation show that the proposed AHMMAS model can effectively detect price manipulation patterns and outperforms the selected benchmark models.

Market abuse has attracted much attention from financial regulators around the world but it is difficult to fully prevent. One of the reasons is the lack of thoroughly studies of the market abuse strategies and the corresponding effective market abuse approaches. In this paper, the strategies of reported price manipulation cases are analysed as well as the related empirical studies. A transformation is then defined to convert the time-varying financial trading data into pseudo-stationary time series, where machine learning algorithms can be easily applied to the detection of the price manipulation. The evaluation experiments conducted on four stocks from NASDAQ show a promising improved performance for effectively detecting such manipulation cases.

Wash trade refers to the activities of traders who utilise deliberately designed collusive transactions to increase the trading volumes for creating active market impression. Wash trade can be damaging to the proper functioning and integrity of capital markets. Existing work focuses on collusive clique detections based on certain assumptions of trading behaviours. Effective approaches for analysing and detecting wash trade in a real-life market have yet to be developed. This paper proposes a new analysis approach for abstracting the basic structures of wash trade based on the network topology theory and a novel approach for detecting wash trade activities. The evaluation experiments conducted on four NASDAQ stocks suggest that wash trade actions can be effectively identified based on the proposed algorithm.

Electronic trading in global markets and exchanges requires sophisticated communication and data management systems. Novel computational infrastructures and trading strategies are required to support the massive amount of incoming streaming data, where the main problem is in latency management. Multi-agent Systems have been recognized as a promising solution to address complex problems in many areas such as biology, social sciences and financial markets and may provide powerful and flexible solutions for implementing trading engines. In addition, reconfigurable hardware based on Field Programmable Gate Arrays (FPGAs) offers many important performance benefits over software implementations, such as reducing decision making latency and high-throughput data processing. Robust and scalable trading engines can be developed by leveraging the benefits of reconfigurable FPGA platforms. This paper presents a multi-agent architecture in reconfigurable hardware for financial applications and the implementation of a trading engine for pre-trade analysis as a validation scenario. Performance results show that calculation of technical indicators and trading strategy evaluation to generate trading signals with a latency of 550 ns is achievable.

In recent years, machine learning algorithms have become increasingly popular in financial forecasting. Their flexible, data-driven nature makes them ideal candidates for dealing with complex financial data. This paper investigates the effectiveness of a number of machine learning algorithms, and combinations of these algorithms, at generating one-step ahead forecasts of a number of financial time series. We find that hybrid models consisting of a linear statistical model and a nonlinear machine learning algorithm are effective at forecasting future values of the series, particularly in terms of the future direction of the series.

Accurate forecasting of directional changes in stock prices is important for algorithmic trading and investment management. Technical analysis has been successfully used in financial forecasting and recently researchers have explored the optimization of parameters for technical indicators. This study investigates the relationship between the window size used for calculating technical indicators and the accuracy of one-step-ahead (variable steps) forecasting. The directions of the future price movements are predicted using technical analysis and machine learning algorithms. Results show a correlation between window size and forecasting step size for the Support Vector Machines approach but not for the other approaches.

Online financial textual information contains a large amount of investor sentiment, i.e. subjective assessment and discussion with respect to financial instruments. An effective solution to automate the sentiment analysis of such large amounts of online financial texts would be extremely beneficial. This paper presents a natural language processing (NLP) based pre-processing approach both for noise removal from raw online financial texts and for organizing such texts into an enhanced format that is more usable for feature extraction. The proposed approach integrates six NLP processing steps, including a developed syntactic and semantic combined negation handling algorithm, to reduce noise in the online informal text. Three-class sentiment classification is also introduced in each system implementation. Experimental results show that the proposed pre-processing approach outperforms other pre-processing methods. The combined negation handling algorithm is also evaluated against three standard negation handling approaches.

Effective robotic grasping and manipulation requires knowledge about the surface properties of an object and the environment in which it is located. Physical contact with materials using tactile sensors can enable the retrieval of detailed information about the material, i.e. compressibility, surface texture and thermal properties. This paper describes a system used to classify a wide range of materials based on their thermal properties and surface texture. Following acquisition of data from a sophisticated tactile sensor, the system uses principal component analysis (PCA) to extract features from the data which are used to train an Artificial Neural Network (ANN) to classify materials, first into groups and then as individual materials. The system is compared with human performance and the results demonstrate that the proposed system performed better than humans by almost 10%.

Machine learning enables the creation of a nonlinear mapping that describes robot-environment interaction, whereas computing linguistics make the interaction transparent. In this paper, we develop a novel application of a linguistic decision tree for a robot route learning problem by dynamically deciding the robot's behavior, which is decomposed into atomic actions in the context of a specified task. We examine the real-time performance of training and control of a linguistic decision tree, and explore the possibility of training a machine learning model in an adaptive system without dual CPUs for parallelization of training and control. A quantified evaluation approach is proposed, and a score is defined for the evaluation of a model's robustness regarding the quality of training data. Compared with the nonlinear system identification nonlinear auto-regressive moving average with eXogeneous inputs model structure with offline parameter estimation, the linguistic decision tree model with online linguistic ID3 learning achieves much better performance, robustness, and reliability.

A self-organising fuzzy-neural network (SOFNN) adapts its structure based on variations of the input data. Conventionally in such self-organising networks, the number of inputs providing the data is fixed. In this paper, we consider the situation where the number of inputs to a network changes dynamically during its online operation. We extend our existing work on a SOFNN such that the SOFNN can self-organise its structure based not only on its input data, but also according to the changes in the number of its inputs. We apply the approach to a smart home application, where there are certain situations when some of the existing events may be removed or new events emerge, and illustrate that our approach enhances cognitive reasoningin a dynamic smart home environment. In this case, the network identifies the removed and/or added events from the received information over time, and reconfigures its structure dynamically. We present results for different combinations of training and testing phases of the dynamic reconfigurable SOFNN using a set of realistic synthesized data. The results show the potential of the proposed method.

The recent development of low cost cameras that capture 3-dimensional images has changed the focus of computer vision research from using solely intensity images to the use of range images, or combinations of RGB, intensity and range images. The low cost and widespread availability of the hardware to capture these images has realised many possible applications in areas such as robotics, object recognition, surveillance, manipulation, navigation and interaction. Given the large volumes of data in range images, processing and extracting the relevant information from the images in real time becomes challenging. To achieve this, much research has been conducted in the area of bio-inspired feature extraction which aims to emulate the biological processes used to extract relevant features, reduce redundancy, and process images efficiently. Inspired by the behaviour of biological vision systems, an approach is presented for extracting important features from intensity and range images, using biologically inspired spiking neural networks in order to model aspects of the functional computational capabilities of the visual system.

Linear time series models, such as the autoregressive integrated moving average (ARIMA) model, are among the most popular statistical models used to forecast time series. In recent years non-linear computational models, such as artificial neural networks (ANN), have been shown to outperform traditional linear models when dealing with complex data, like financial time series. This paper proposes a novel hybrid forecasting model which exploits the linear modelling strengths of the ARIMA model, and the flexibility of a self-organising fuzzy neural network (SOFNN). The system's performance is evaluated using several datasets, and our results indicate that a hybrid system is an effective tool for time series forecasting.

Price manipulation refers to the act of using illegal trading behaviour to manually change an equity price with the aim of making profits. With increasing volumes of trading, price manipulation can be extremely damaging to the proper functioning and integrity of capital markets. Effective approaches for analysing and real-time detection of price manipulation are yet to be developed. This paper proposes a novel approach, called Hidden Markov Model with Abnormal States (HMMAS), which models and detects price manipulation activities. Together with the wavelet decomposition for features extraction and Gaussian Mixture Model for Probability Density Function (PDF) construction, the HMMAS model detects price manipulation and identifies the type of the detected manipulation. Evaluation experiments of the model were conducted on six stock tick data from NASDAQ and London Stock Exchange (LSE). The results showed that the proposed HMMAS model can effectively detect price manipulation patterns.

Many robots use range data to obtain an almost 3-dimensional description of their environment. Feature driven segmentation of range images has been primarily used for 3D object recognition, and hence the accuracy of the detected features is a prominent issue. Inspired by the structure and behaviour of the human visual system, we present an approach to feature extraction in range data using spiking neural networks and a biologically plausible hexagonal pixel arrangement. Standard digital images are converted into a hexagonal pixel representation and then processed using a spiking neural network with hexagonal shaped receptive fields; this approach is a step towards developing a robotic eye that closely mimics the human eye. The performance is compared with receptive fields implemented on standard rectangular images. Results illustrate that, using hexagonally shaped receptive fields, performance is improved over standard rectangular shaped receptive fields.

This paper presents a position based visual tracking system of a redundant manipulator using a Kinect camera. Kinect camera provides 3-D information of a target object, therefore the control algorithm of the position-based visual servoing (PBVS) can be simplified, as there is no requirement to estimate a 3-D feature point position from the extracted image and the camera model. The Kalman filter is used to predict the target position and velocity. This control method is applied to a calibrated robotic system with eye-to-hand configuration. The stability analysis has been derived and real-time experiments have been carried out using a 7 DOF PowerCube manipulator from Amtec Robotic. The experimental results of both static and moving targets are presented to demonstrate and to verify the proposed position based visual tracking system performance.

Corner detection is used in many computer vision applications that require fast and efficient feature matching. In addition, hexagonal pixel based images have been recently investigated for image capture and processing due to their ability to represent curved structures that are common in real images better than traditional rectangular pixel based images. Therefore, we present an approach to corner detection on hexagonal images and demonstrate that accuracy is comparable to well-known existing corner detectors applied to rectangular pixel based images.

We present a biologically motivated approach to fast feature extraction on hexagonal pixel based images using the concept of eye tremor in combination with the use of the spiral architecture and convolution of non-overlapping gradient masks. We generate seven feature maps “a-trous” that can be combined into a single complete feature map, and we demonstrate that this approach is significantly faster than the use of conventional spiral convolution or the use of a neighbourhood address look-up table on hexagonal images.

In a multi-robot system, the coordination and cooperation among the robots determine the effectiveness of task execution. Different centralised and distributed task allocation algorithms have been proposed by researchers. Recently consensus based task allocation has been extensively researched because of its robustness in handling large teams of robots. We propose a new auction and consensus based algorithm for fast task allocation in parallel with task execution. The performance of the proposed algorithm under different conditions is analyzed and compared with other distributed consensus algorithms.

Inspired by the structure and behaviour of the human visual system, we present an approach to edge detection using spiking neural networks and a biologically plausible hexagonal pixel arrangement. Standard digital images are converted into a hexagonal pixel representation and then processed using a spiking neural network with hexagonal shaped receptive fields. The performance is compared with receptive fields implemented on standard rectangular images. Results illustrate that, using hexagonal shaped receptive fields, performance is improved over standard rectangular shaped receptive fields.

This paper presents a distributed fuzzy proportional control system for a vision guided redundant robot manipulator. Firstly, the Takagi Sugeno (TS) fuzzy algorithm is used to model analytical Image Based Visual Servoing (IBVS) using shape moments by offline learning. This control method is applied to an uncalibrated robotic system with eye-in-hand visual feedback. The system is able to track a moving object through a variety of motions and maintain the object's image features in a desired position in the image plane without a priori knowledge of the robot kinematic, camera calibration and inverse Jacobian. The experimental results of both static and moving targets using the 7 DOF PowerCube manipulator from Amtec Robotic show and verify its performance in a realtime application.

A systematic design procedure is used to develop Laplacian operators that facilitate the computation of hexagonal feature map pyramids. Our focus is the development of algorithms that can operate on hexagonal images over a range of scales. We show how scalable operators can be explicitly constructed using a Gaussian neighbourhood function. We extend this approach to achieve an efficient approximation via a feature map pyramid that implicitly embodies operator scaling. In both cases we provide performance evaluation with respect to edge localisation.

We present a general approach to the computation of adaptive tri-directional operators for use on hexagonal pixel-based images, based on the spiral architecture. We show that the use of Gaussian basis functions within the finite element method provides a framework for a systematic design procedure for operators that are adaptive to spiral neighbourhoods through the use of an explicit scale parameter. We evaluate the proposed operators using simulated hexagonal images and provide comparative results with the use of traditional rectangular operators.

The Spiral Architecture has been developed as a fast way of indexing a hexagonal pixel-based image. In combination with spiral addition and spiral multiplication, methods have been developed for hexagonal image processing operations such as translation and rotation. Using the Spiral Architecture as the basis for our operator structure, we present a general approach to the computation of adaptive coarse scale Laplacian operators for use on hexagonal pixel-based images. We evaluate the proposed operators using simulated hexagonal images and demonstrate improved performance when compared with rectangular Laplacian operators such as Marr-Hildreth.

Feature extraction in image data has been investigated for many years, and more recently the problem of processing images containing irregularly distributed data has become prominent. Range data are now commonly used in the areas of image processing and computer vision. However, due to the data irregularity found in range images that occurs with a variety of image sensors, direct image processing, in particular edge detection, is a non-trivial problem. Typically, irregular range data would require to be interpolated to a regular grid prior to processing. One example of an edge detection technique than can be directly applied to range images is the scan-line approximation, but this does not employ exact data locations. Therefore, we present novel Laplacian operators that can be applied directly to irregularly distributed data, and in particular we focus on application to irregularly distributed 3D range data for the purpose of edge detection. Within the data distribution framework commonly occurring in range data acquisition devices, our results illustrate that the approach works well over a range of levels of irregularity of data distribution. The use of Laplacian operators on range data is also found to be much less susceptible to noise than the traditional use of Laplacian operators on intensity images.

Use of incomplete image data has become a prominent research issue in recent years, driven by the development of space variant image sensors. Whilst image reconstruction techniques have been developed that enable the subsequent use of standard image processing algorithms, the development of image processing algorithms that can be applied directly to incomplete image data has received less attention. The problem of interest point detection for incomplete images is addressed by presenting an algorithm that can be applied directly to incomplete image data without the requirement of image reconstruction, and the accurate performance of the algorithm is illustrated through visual results and ROC curves.

To enable fast reliable feature matching or tracking in scenes, features need to be discrete and meaningful, and hence corner detection is often used for this purpose. However, to obtain a higher level description of an image, such as identification of objects, additional information such as edges is required, and more recently detectors have been proposed that find both edges and corners. We present a combined operator, enabling edge and corner detection to be achieved concurrently. We demonstrate that accuracy is comparable to well-known existing corner detectors and edge detectors, and, as standard post-smoothing of the corner map is not required, significantly reduced computation time can be achieved.

The use of range data has become prominent in the field of computer vision. Due to the irregular nature of range data that occurs with a number of sensors, feature extraction is a complex and challenging problem. Feature extraction techniques for range images are often based on scan line data approximations and hence do not employ exact data locations. We present a finite element based approach to the development of Laplacian operators that can be applied to both regularly or irregularly distributed range data. We demonstrate that the feature maps generated using our approach on range data are much less susceptible to noise than the traditional use of Laplacian operators on intensity images.

Range images can provide an almost 3-dimensional description of a scene. Feature driven segmentation of range images has been primarily used for 3D object recognition, and hence the accuracy of the detected features is a prominent issue. Feature extraction on range images has proven to be a more complex problem than on intensity images due to both the irregular distribution of range image data and the nature of the features that are present in range images. Approaches to range image feature extraction are often scan line based approximations that carry a significant computational overhead and hence are not appropriate for real-time processing. This paper presents a design procedure for scalable first order derivative operators that can be used directly on irregularly distributed data. Hence the method is appropriate for direct use on range image data without the requirement of image preprocessing and could form the basis of algorithms of real-time robotic applications.

The use of omnidirectional cameras has had a significant impact on the success of vision systems for video surveillance and autonomous robot navigation. Typically images obtained from such cameras are transformed to sparse panoramic images that are interpolated prior to low level image processing. We present a graph theoretic approach that enables image processing techniques, principally feature extraction, to be performed directly on sparse panoramic images, avoiding the need for image interpolation. We thus aim to reduce the computational overheads of processing images arising from omnidirectional cameras, whilst retaining accuracy sufficient for application to real-time robot vision.

Incomplete image data sets are of interest in many domains and arise in a variety of applications, and in particular in applications that use remote sensor array data. Although recent developments in mesh modelling of images have provided algorithms that can achieve accurate and efficient image representations without the high computational cost associated with earlier optimisation-based methods, such techniques rely on the availability of the entire image data. These content-based mesh modelling techniques aim to provide a high sample density in regions of interest, such as feature neighbourhoods or around moving objects, whilst achieving efficiency by retaining a low overall image sampling density. The sampling density is determined by a feature map, such as local image curvature or local spatial-frequency content that is obtained from the underlying complete image data. As the requirement for the availability of complete image data makes such content-based mesh modelling techniques unsuitable for application to incomplete images, where an image consists of a sparse data set, we aim to address this issue by proposing an alternative approach to mesh modelling that is based on automatically adaptive feature detection directly applicable to sparsely sampled images.

Recent developments in mesh modelling of images have provided algorithms that can achieve accurate and efficient image representations without the high computational cost associated with earlier optimisation-based methods. Hence nonuniform sampling of images combined with the use of irregular content-based meshing has provided a successful basis for recent developments in image compression techniques. The evaluation of these techniques has focussed on the accuracy and efficiency with which the mesh model can represent the image. For real-time applications, the usefulness of a mesh model may be assessed by its ability to yield compressed image representations that can be processed directly to provide output that is sufficiently accurate. Hence we present an evaluation of mesh model algorithms that is based on feature detection on the associated compressed image representations. Such an approach is built on the recent development of systematic design procedures for scalable and adaptive image processing operators that can be applied directly to non-uniformly sampled images. We demonstrate the approach using image derivative operators on compressed images.

Non-uniformly sampled images represented on irregular content-based meshes are central to the developments in image compression techniques and in efficient motion tracking. We present a general approach to the development of systematic design procedures for scalable and adaptive low level image processing operators that can be applied to such non-uniformly sampled images. We provide algorithms that use the content-based mesh to address the usually difficult issue of local operator scale selection. The operator scale is therefore automatically matched to the local scale of the image features as embodied in the mesh. In this way we are able to apply a range of operators directly to compressed images. We demonstrate the approach with the design of image derivative operators that enable image feature detection to be implemented directly on compressed images.

In image filtering, the 'circularity' of an operator is an important factor affecting its accuracy. For example, circular differential edge operators are effective in minimising the angular error in the estimation of image gradient direction. We present a general approach to the computation of scalable circular low-level image processing operators that is based on the finite element method. We show that the use of Gaussian basis functions within the finite element method provides a framework for a systematic and efficient design procedure for operators that are scalable to near-circular neighbourhoods through the use of an explicit scale parameter. The general design technique may be applied to a range of operators. Here we evaluate the approach for the design of an image gradient operator, and we present comparative results with other gradient approximation methods.