Expressing emotions is essential in human interaction.Often, individuals convey emotions through neutral speech, while the underlying meaning carries emotional weight.Conversely, tone can also convey emotion despite neutral words.Most Speech Emotion Recognition research overlooks this.We address this gap with a multimodal emotion recognition system using hierarchical classifiers and a novel decision fusion method.Our approach analyses emotional cues from speech and text, measuring their impact on predicted classes, considering emotional or neutral contributions for each instance.Results on the IEMOCAP dataset show our method's effectiveness: 69.45% and 65.26% weighted accuracy in speaker-dependent and speaker-independent settings, respectively.

Machine learning models have significantly enriched the toolbox in the field of neuroimaging analysis. Among them, Support Vector Machines (SVM) have been one of the most popular models for supervised learning, but their use primarily relies on linear SVM models due to their explainability. Kernel SVM models are capable classifiers but more opaque. Recent advances in eXplainable AI (XAI) have developed several feature importance methods to address the explainability problem. However, noise variables can affect these explanations, making irrelevant variables regarded as important variables. This problem also appears in explaining linear models, which the linear pattern can address. This paper proposes a fast method to explain RBF kernel SVM globally by adopting the notion of linear pattern in kernel space. Our method can generate global explanations with low computational cost and is less affected by noise variables. We successfully evaluate our method on simulated and real MEG/EEG datasets.

Deep-learning-based single-image super-resolution models are typically trained using image patches, rather than the whole images, due to hardware limits. Since different brain regions have disparate structures and their size varies, such as the cerebrum and cerebellum, models trained using image patches can be dominated by the structures of the larger brain regions and ignore the fine-grained details in smaller areas. In this paper, we first evaluate several previously proposed models using more blurry low-resolution images than previous studies, as input. Then, we propose an effective approach for the conventional patch-based strategy by balancing the proportion of patches containing high-frequency details. This makes the model focus more on high-frequency information in tiny regions, especially for the cerebellum. Compared with the conventional patch-based strategy, the resultant super-resolved image from our approach achieves comparable image quality in the whole brain. In contrast, it improves significantly on the high-frequency details in the cerebellum.

Machine learning methods have shown great performance in many areas, including neuroimaging data analysis. However, model performance is only one objective in neuroimaging analysis. Gaining insight from the data is also critical in this field, such as identifying regions where detected signals are relevant to cognitive and diagnostic tasks. To fulfil this need, enabling the explainability of a model’s decision-making process is critical. Predictions of complex machine learning models are notoriously difficult to explain. This limits the use of complex models like kernel support vector machines (SVM) in neuroimaging analysis. Recently, several permutation-based methods have been developed to explain these complex models. However, the explanation results are affected by class-irrelevant features like suppressor variables and high background noise variables. This problem may also happen when explaining linear models. One possible reason is that the permutation process will produce unrealistic data instances when features are not independent, e.g. correlated. These unrealistic data instances will influence the explanation results. In neuroimaging analysis, the activation pattern, the estimated weight of the assumed generative model corresponding to the current classifier, is used to deal with this problem for linear models. This method does not rely on a permutation process but rather on the available data information. In this paper, we propose a novel method of Explanation through Activation Pattern (EAP) to explain the SVM models with different types of kernels for neuroimaging data analysis. Our method can generate a global feature importance score by estimating the activation pattern of kernel SVM models. We evaluate our method against three popular methods on both simulation datasets and a publicly available EEG/MEG dataset on visual tasks. The experimental results demonstrate that the proposed EAP method can provide explanations with low computational cost and is less affected by class-irrelevant features than the other three methods. In the experiment using the MEG/EEG dataset of visual tasks, the proposed EAP method provides agreement results with the brain’s electrical activity patterns reported in the literature on the visual tasks EEG/MEG data and is significantly faster than the other explanation methods.

Previous studies have shown that there is a strong correlation between radiologists' diagnoses and their gaze when reading medical images. The extent to which gaze is attracted by content in a visual scene can be characterised as visual saliency. There is a potential for the use of visual saliency in computer-aided diagnosis in radiology. However, little is known about what methods are effective for diagnostic images, and how these methods could be adapted to address specific applications in diagnostic imaging. In this study, we investigate 20 state-of-the-art saliency models including 10 traditional models and 10 deep learning-based models in predicting radiologists' visual attention while reading 196 mammograms. We found that deep learning-based models represent the most effective type of methods for predicting radiologists' gaze in mammogram reading; and that the performance of these saliency models can be significantly improved by transfer learning. In particular, an enhanced model can be achieved by pre-training the model on a large-scale natural image saliency dataset and then fine-tuning it on the target medical image dataset. In addition, based on a systematic selection of backbone networks and network architectures, we proposed a parallel multi-stream encoded model which outperforms the state-of-the-art approaches for predicting saliency of mammograms.

In neuroimaging, the difference between predicted brain age and chronological age, known as brain age delta, has shown its potential as a biomarker related to various pathological phenotypes. There is a frequently observed bias when estimating brain age delta using regression models. This bias manifests as an overestimation of brain age for young participants and an underestimation of brain age for older participants. Therefore, the brain age delta is negatively correlated with chronological age, which can be problematic when evaluating relationships between brain age delta and other age-associated variables. This paper proposes a novel bias correction method for regression models by introducing a skewed loss function to replace the normal symmetric loss function. The regression model then behaves differently depending on whether it makes overestimations or underestimations. Our approach works with any type of MR image and no specific preprocessing is required, as long as the image is sensitive to age-related changes. The proposed approach has been validated using three classic deep learning models, namely ResNet, VGG, and GoogleNet on publicly available neuroimaging aging datasets. It shows flexibility across different model architectures and different choices of hyperparameters. The corrected brain age delta from our approach then has no linear relationship with chronological age and achieves higher predictive accuracy than a commonly-used two-stage approach.

Radiologists’ eye-movement during diagnostic image reading reflects their personal training and experience, which means that their diagnostic decisions are related to their perceptual processes. For training, monitoring, and performance evaluation of radiologists, it would be beneficial to be able to automatically predict the spatial distribution of the radiologist’s visual attention on the diagnostic images. The measurement of visual saliency is a well-studied area that allows for prediction of a person’s gaze attention. However, compared with the extensively studied natural image visual saliency (in free viewing tasks), the saliency for diagnostic images is less studied; there could be fundamental differences in eye-movement behaviours between these two domains. Most current saliency prediction models have been optimally developed for natural images, which could lead them to be less adept at predicting the visual attention of radiologists during the diagnosis. In this paper, we propose a method specifically for automatically capturing the visual attention of radiologists during mammogram reading. By adopting high-resolution image representations from both deep and shallow encoders, the proposed method avoids potential detail losses and achieves superior results on multiple evaluation metrics in a large mammogram eye-movement dataset.

CNN architectures that take videos as an input are often overlooked when it comes to the development of explanation techniques. This is despite their use in often critical domains such as surveillance and healthcare. Explanation techniques developed for these networks must take into account the additional temporal domain if they are to be successful. In this paper we introduce SWAG-V, an extension of SWAG for use with networks that take video as an input. In addition we show how these explanations can be created in such a way that they are balanced between fine and coarse explanations. By creating superpixels that incorporate the frames of the input video we are able to create explanations that better locate regions of the input that are important to the networks prediction. We compare SWAG-V against a number of similar techniques using metrics such as insertion and deletion, and weak localisation. We compute these using Kinetics-400 with both the C3D and R(2+1)D network architectures and find that SWAG-V is able to outperform multiple techniques.

Convolutional neural networks (CNNs) have significantly advanced computational modelling for saliency prediction. However, accurately simulating the mechanisms of visual attention in the human cortex remains an academic challenge. It is critical to integrate properties of human vision into the design of CNN architectures, leading to perceptually more relevant saliency prediction. Due to the inherent inductive biases of CNN architectures, there is a lack of sufficient long-range contextual encoding capacity. This hinders CNN-based saliency models from capturing properties that emulate viewing behaviour of humans. Transformers have shown great potential in encoding long-range information by leveraging the self-attention mechanism. In this paper, we propose a novel saliency model that integrates transformer components to CNNs to capture the long-range contextual visual information. Experimental results show that the transformers provide added value to saliency prediction, enhancing its perceptual relevance in the performance. Our proposed saliency model using transformers has achieved superior results on public benchmarks and competitions for saliency prediction models. The source code of our proposed saliency model TranSalNet is available at: https://github.com/LJOVO/TranSalNet.

Providing an explanation of the operation of CNNs that is both accurate and interpretable is becoming essential in fields like medical image analysis, surveillance, and autonomous driving. In these areas, it is important to have confidence that the CNN is working as expected and explanations from saliency maps provide an efficient way of doing this. In this paper, we propose a pair of complementary contributions that improve upon the state of the art for region-based explanations in both accuracy and utility. The first is SWAG, a method for generating accurate explanations quickly using superpixels for discriminative regions which is meant to be a more accurate, efficient, and tunable drop in replacement method for Grad-CAM, LIME, or other region-based methods. The second contribution is based on an investigation into how to best generate the superpixels used to represent the features found within the image. Using SWAG, we compare using superpixels created from the image, a combination of the image and backpropagated gradients, and the gradients themselves. To the best of our knowledge, this is the first method proposed to generate explanations using superpixels explicitly created to represent the discriminative features important to the network. To compare we use both ImageNet and challenging fine-grained datasets over a range of metrics. We demonstrate experimentally that our methods provide the best local and global accuracy compared to Grad-CAM, Grad-CAM++, LIME, XRAI, and RISE.

We propose a novel model for temporal detection and localization which allows the training of deep neural networks using only counts of event occurrences as training labels. This powerful weakly-supervised framework alleviates the burden of the imprecise and time consuming process of annotating event locations in temporal data. Unlike existing methods, in which localization is explicitly achieved by design, our model learns localization implicitly as a byproduct of learning to count instances. This unique feature is a direct consequence of the model’s theoretical properties. We validate the effectiveness of our approach in a number of experiments (drum hit and piano onset detection in audio, digit detection in images) and demonstrate performance comparable to that of fully-supervised state-of-the-art methods, despite much weaker training requirements.

The relationship between the shape and gender of a face, with particular application to automatic gender classification, has been the subject of significant research in recent years. Determining the gender of a face, especially when dealing with unseen examples, presents a major challenge. This is especially true for certain age groups, such as teenagers, due to their rapid development at this phase of life. This study proposes a new set of facial morphological descriptors, based on 3D geodesic path curvatures, and uses them for gender analysis. Their goal is to discern key facial areas related to gender, specifically suited to the task of gender classification. These new curvature-based features are extracted along the geodesic path between two biological landmarks located in key facial areas. Classification performance based on the new features is compared with that achieved using the Euclidean and geodesic distance measures traditionally used in gender analysis and classification. Five different experiments were conducted on a large teenage face database (4745 faces from the Avon Longitudinal Study of Parents and Children) to investigate and justify the use of the proposed curvature features. Our experiments show that the combination of the new features with geodesic distances provides a classification accuracy of 89%. They also show that nose-related traits provide the most discriminative facial feature for gender classification, with the most discriminative features lying along the 3D face profile curve.

The allocation of resources to challenge city centre violent crime traditionally relies on historical data to identify hot-spots. The usefulness of such data-driven approaches is limited when historical data is scarce or unavailable (e.g. planning of a new city) or insufficiently representative (e.g. does not account for novel events, such as Olympic Games). In some cities, crime data is not systematically accumulated at all. We present a graph-constrained agent based simulation model of alcohol-related violent crime that is capable of predicting areas of likely violent crime without requiring any historical data. The only inputs to our simulation are publicly available geographical data, which makes our method immediately applicable to a wide range of tasks, such as optimal city planning, police patrol optimisation, devising alcohol licensing policies. In experiments, we evaluate our model and demonstrate agreement of our model's predictions on where and when violence will occur with real-world violent crime data. Analyses indicate that our agent based model may be able to make a significant contribution to attempts to prevent violence through deterrence or by design.

Facial ageing analysis based on 4D data (3D plus time) is much more robust to pose changes and illumination variations than using 2D image and video. The purpose of this investigation was to measure the effects of age and gender related facial changes using dynamic 3D facial scans. Experiments were carried out on the subjects, who were divided into two groups by age (15-30 years and 31-60 years). Each group was further subdivided by gender. 3D scans of the subjects were processed to extract facial features which were tracked through the duration of the data capture. Subsequently, a set of dynamic features were computed from these facial features, as well as static features for comparison. Two-way multivariate analysis of variance (MANOVA) of these features demonstrated that statistically significant age and gender related differences could be detected. We show that 3D facial dynamics provide more useful information than static features for the characterisation of smiles.

The severity of sustained injury resulting from assault-related violence can be minimised by reducing detection time. However, it has been shown that human operators perform poorly at detecting events found in video footage when presented with simultaneous feeds. We utilise computer vision techniques to develop an automated method of abnormal crowd detection that can aid a human operator in the detection of violent behaviour. We observed that behaviour in city centre environments often occurs in crowded areas, resulting in individual actions being occluded by other crowd members. We propose a real-time descriptor that models crowd dynamics by encoding changes in crowd texture using temporal summaries of grey level co-occurrence matrix features. We introduce a measure of inter-frame uniformity and demonstrate that the appearance of violent behaviour changes in a less uniform manner when compared to other types of crowd behaviour. Our proposed method is computationally cheap and offers real-time description. Evaluating our method using a privately held CCTV dataset and the publicly available Violent Flows, UCF Web Abnormality and UMN Abnormal Crowd datasets, we report a receiver operating characteristic score of 0.9782, 0.9403, 0.8218 and 0.9956, respectively.

In this paper we introduce a novel approach for building 4D coupled statistical models of conversational facial expression interactions. To build these coupled models we use 3D AAMs for feature extraction, 4D polynomial fitting for sequence representation, and concatenated feature vectors of frontchannel-backchannel interactions (with offset values) for the coupled model. Using a coupled model of conversation smile interactions, we predicted each sequence’s backchannel signal. In a subsequent experiment, human observers rated predicted sequences as highly similar to the originals. Our results demonstrate the usefulness of coupled models as powerful tools to analyse and synthesise key aspects of conversational interactions, including conversation timings, backchannel responses to frontchannel signals, and the spatial and temporal dynamics of conversational facial expression interactions.

Facial morphology measurement and classification play important role in the face anthropometry of many medical applications. This usually involves the investigation of medical abnormalities where specific facial features are studied by taking a number of measurements of the facial area under investigation. The measurements are often obtained from the three-dimensional (3D) scans of the faces; however, the measurements are often made manually, which is tedious and time consuming process. Moreover, in gene related studies thousands of measurements may be necessary in order to find statistically significant relationships between facial features and genes. Normative studies, from which typical populous models can be built, also require many measurements. Thus an automatic method to extract morphological measurements and interpret them is desirable. In this article, an automatic method for classification of facial morphology on the basis of a number of geometric measurements obtained automatically from 3D facial scans is presented. Among different facial features the philtrum, which is the vertical groove extending from the nose to the upper lip and the lip area, plays an important role in defining the interaction between the genes and craniofacial anomalies such as, for example, cleft lip and palate. In this paper, geometric features are analysed for their suitability to classify philtrum into three classes previously proposed by medical experts. Moreover, further analysis is conducted to assess the best number of classes to model the underlying data distribution from the point of view of classification accuracy. The obtained classification results are compared with the ground truth manual labelling of 3D face meshes provided by a medical expert. The dataset used for this research is taken from ALSPAC dataset and consists of 1000 3D face meshes. The proposed method achieves classification accuracy of 97% for this data set using the Mean, Minimum and Maximum curvature features in combination.

Abstract We present a new real‐time approach to simulate deformable objects using a learnt statistical model to achieve a high degree of realism. Our approach improves upon state‐of‐the‐art interactive shape‐matching meshless simulation methods by not only capturing important nuances of an object's kinematics but also of its dynamic texture variation. We are able to achieve this in an automated pipeline from data capture to simulation. Our system allows for the capture of idiosyncratic characteristics of an object's dynamics which for many simulations (e.g. facial animation) is essential. We allow for the plausible simulation of mechanically complex objects without knowledge of their inner workings. The main idea of our approach is to use a flexible statistical model to achieve a geometrically‐driven simulation that allows for arbitrarily complex yet easily learned deformations while at the same time preserving the desirable properties (stability, speed and memory efficiency) of current shape‐matching simulation systems. The principal advantage of our approach is the ease with which a pseudo‐mechanical model can be learned from 3D scanner data to yield realistic animation. We present examples of non‐trivial biomechanical objects simulated on a desktop machine in real‐time, demonstrating superior realism over current geometrically motivated simulation techniques.

Natural image statistics is an important area of research in cognitive sciences and computer vision. Visualization of statistical results can help identify clusters and anomalies as well as analyze deviation, distribution, and correlation. Furthermore, they can provide visual abstractions and symbolism for categorized data. In this paper, we begin our study of visualization of image statistics by considering visual representations of power spectra, which are commonly used to visualize different categories of images. We show that they convey a limited amount of statistical information about image categories and their support for analytical tasks is ineffective. We then introduce several new visual representations, which convey different or more information about image statistics. We apply ANOVA to the image statistics to help select statistically more meaningful measurements in our design process. A task-based user evaluation was carried out to compare the new visual representations with the conventional power spectra plots. Based on the results of the evaluation, we made further improvement of visualizations by introducing composite visual representations of image statistics.

We introduce a video-based approach for producing water surface models. Recent advances in this field output high-quality results but require dedicated capturing devices and only work in limited conditions. In contrast, our method achieves a good tradeoff between the visual quality and the production cost: It automatically produces a visually plausible animation using a single viewpoint video as the input. Our approach is based on two discoveries: first, shape from shading (SFS) is adequate to capture the appearance and dynamic behavior of the example water; second, shallow water model can be used to estimate a velocity field that produces complex surface dynamics. We will provide qualitative evaluation of our method and demonstrate its good performance across a wide range of scenes.

Three-dimensional surface registration transforms multiple three-dimensional data sets into the same coordinate system so as to align overlapping components of these sets. Recent surveys have covered different aspects of either rigid or nonrigid registration, but seldom discuss them as a whole. Our study serves two purposes: 1) To give a comprehensive survey of both types of registration, focusing on three-dimensional point clouds and meshes and 2) to provide a better understanding of registration from the perspective of data fitting. Registration is closely related to data fitting in which it comprises three core interwoven components: model selection, correspondences and constraints, and optimization. Study of these components 1) provides a basis for comparison of the novelties of different techniques, 2) reveals the similarity of rigid and nonrigid registration in terms of problem representations, and 3) shows how overfitting arises in nonrigid registration and the reasons for increasing interest in intrinsic techniques. We further summarize some practical issues of registration which include initializations and evaluations, and discuss some of our own observations, insights and foreseeable research trends.

We consider the problem of producing accurate facial animation corresponding to a given input speech signal. A popular technique previously used for Audio Driven Facial Animation is to build a joint audio-visual model using Active Appearance Models (AAMs) to represent possible facial variations and Hidden Markov Models (HMMs) to select the correct appearance based on the input audio. However there are several questions that remained unanswered. In particular the choice of clustering technique and the choice of the number of clusters in the HMM may have significant influence over the quality of the produced videos. We have investigated a range of clustering techniques in order to improve the quality of the HMM produced, and proposed a new structure based on using Gaussian Mixture Models (GMMs) to model each phoneme separately. We compared our approach to several alternatives using a public dataset of 300 phonetically labeled sentences spoken by a single person and found that our approach produces more accurate animation. In addition, we use a hybrid approach where the training data is phonetically labeled thus producing a model with better separation of phonemes, but test audio data is not labeled, thus making our approach for generating facial animation less laborious and fully automatic.

In this paper we introduce a framework for the segmentation of scanned scrolled parchments, based on a novel graph cut based approach with an additional shape prior, in combination with anisotropic diffusion and geometry-constrained postprocessing. This problem has not been investigated by the computer vision community properly yet due to the parchment scanning technology novelty, and is extremely important for effective data recovery from historical scrolled documents whose content is inaccessible due to the deterioration of the parchment. To date, parchment segmentation has required user interaction, which is very time consuming for such data. We demonstrate with real examples how our algorithm is able to solve the major problem for scrolled parchment analysis, namely segment connected layers, and process the data without user interaction.

Advances in 3D imaging have recently made 3D surface scanners, capable of capturing textured surfaces at video rate, affordable and common in computer vision. This is a relatively new source of data, the potential of which has not yet been fully exploited as the problem of non-rigid registration of surfaces is difficult. While registration based on shape alone has been an active research area for some time, the problem of registering surfaces based on texture information has not been addressed in a principled way. We propose a novel, efficient and reliable, fully automatic method for performing groupwise non-rigid registration of textured surfaces, such as those obtained with 3D scanners. We demonstrate the robustness of our approach on 3D scans of human faces, including the notoriously difficult case of inter-subject registration. We show how our method can be used to build high-quality 3D models of appearance fully automatically.

This paper describes a novel approach to building models of temporal dynamics for facial animation with applications in performing perceptual testing of trustworthiness. A vital component of the system is a method to bring two image sequences into temporal alignment. Our approach is to project the two sequences into face space (built using shape models [1]) and apply dynamic time warping (DTW). However, the variability in the sequences causes the standard DTW algorithm to perform poorly on our data, and so we have overcome this by extending DTW in the following ways: 1) the signal magnitudes are augmented by incorporating derivatives [2], and a scheme for estimating weights in the cost function is proposed, 2) the set of sequences is used to build a graph, with nodes representing sequences and edges indicating the cost of applying the extended DTW to align pairs of sequences; better alignments between sequences can now be found by traversing the minimum cost path through the graph. Once all signals are aligned to a common temporal reference it is straightforward to map the temporal dynamics from one face to another. A remapped face is synthesised using the new trajectory in face space to drive an active appearance model [1]. Furthermore, the common temporal reference allows us to build a statistical model of the dynamics. This can be used to both identify dynamics of interest and also to manipulate the dynamics, e.g. to reduce or exaggerate facial dynamics.

In this paper we investigate the challenge of 3D reconstruction from Snooker video data. We propose a system pipeline for intelligent filtering based on semantic importance in Snooker. The system can be divided into table detection and correction, followed by ball detection, classification and tracking. It is apparent from previous work that there are several challenges presented here. Firstly, previous methods tend to use a fixed top-down camera mounted above the table. To capture a full table view from this is challenging due to space limitations above the table. Instead, we capture video data from a tripod and correct the viewpoint through processing. Secondly, previous methods tend to simply detect the balls without considering other interfering objects such as player and cue. This becomes even more apparent when the player strikes the cue ball. Our intelligent filtering avoids such issues to give accurate 3D table reconstruction.

We propose a novel system that aids engineers in the conceptual stage of design. Our system's goal is to support the engineer without limiting his creative role; thus, our proposed method does not produce ready study solutions but rather actively monitors the design procedure, verifying design stages and pointing out potential mistakes. This is achieved with a hybrid computer vision, machine learning and knowledge based system framework. Design stage identification is performed with a novel algorithm which comprises a classification stage based on Random Forests and examination of the temporal relationships between the engineer's actions with the aid of statistical graphical models. Experimental results captured in a complex, real life scenario demonstrate our system's ability to efficiently support the engineer's decisions during the conceptual stage of design.

Abstract Over the past decade, computer scientists and psychologists have made great efforts to collect and analyze facial dynamics data that exhibit different expressions and emotions. Such data is commonly captured as videos and are transformed into feature‐based time‐series prior to any analysis. However, the analytical tasks, such as expression classification, have been hindered by the lack of understanding of the complex data space and the associated algorithm space. Conventional graph‐based time‐series visualization is also found inadequate to support such tasks. In this work, we adopt a visual analytics approach by visualizing the correlation between the algorithm space and our goal – classifying facial dynamics. We transform multiple feature‐based time‐series for each expression in measurement space to a multi‐dimensional representation in parameter space. This enables us to utilize parallel coordinates visualization to gain an understanding of the algorithm space, providing a fast and cost‐effective means to support the design of analytical algorithms.

Although the human face is commonly used as a physiological biometric, very little work has been done to exploit the idiosyncrasies of facial motions for person identification. In this paper, we investigate theuniquenessandpermanenceof facial actions to determine whether these can be used as a behavioral biometric. Experiments are carried out using 3-D video data of participants performing a set of very short verbal and nonverbal facial actions. The data have been collected over long time intervals to assess the variability of the subjects' emotional and physical conditions. Quantitative evaluations are performed for both the identification and the verification problems; the results indicate that emotional expressions (e.g., smile and disgust) are not sufficiently reliable for identity recognition in real-life situations, whereas speech-related facial movements show promising potential.

The groupwise approach to non-rigid image registration, solving the dense correspondence problem, has recently been shown to be a useful tool in many applications, including medical imaging, automatic construction of statistical models of appearance and analysis of facial dynamics. Such an approach overcomes limitations of traditional pairwise methods but at a cost of having to search for the solution (optimal registration) in a space of much higher dimensionality which grows rapidly with the number of examples (images) being registered. Techniques to overcome this dimensionality problem have not been addressed sufficiently in the groupwise registration literature. In this paper, we propose a novel, fast and reliable, fully unsupervised stochastic algorithm to search for optimal groupwise dense correspondence in large sets of unmarked images. The efficiency of our approach stems from novel dimensionality reduction techniques specific to the problem of groupwise image registration and from comparative insensitivity of the adopted optimisation scheme (simultaneous perturbation stochastic approximation (SPSA)) to the high dimensionality of the search space. Additionally, our algorithm is formulated in way readily suited to implementation on graphics processing units (GPU). In evaluation of our method we show a high robustness and success rate, fast convergence on various types of test data, including facial images featuring large degrees of both inter- and intra-person variation, and show considerable improvement in terms of accuracy of solution and speed compared to traditional methods.

This paper investigates the use of facial gestures for identity recognition. This is the first time that such a quantitative evaluation is conducted, comparing the analyses of 2D versus 3D dynamic data of verbal and nonverbal facial actions. Suitable data processing and feature extraction methods are examined, then a number of pattern matching techniques including the Fréchet distance, Correlation Coefficients, Hidden-Markov Models, Dynamic Time Warping and its derived forms are compared, in light of which an improved algorithm is proposed. Finally, a face recognition prototype using facial dynamics is built, achieving an Equal Error Rate EER=1.6%. 1

Motivated by the need for an informative, unbiased, and quantitative perceptual method for the evaluation of a talking head we are developing, we propose a new test based on the “McGurk Effect.” Our approach helps to identify strengths and weaknesses in visual--speech synthesis algorithms for talking heads and facial animations, in general, and uses this insight to guide further development. We also evaluate the behavioral quality of our facial animations in comparison to real-speaker footage and demonstrate our tests by applying them to our current speech-driven facial animation system.

Reverse engineering allows the geometric reconstruction of simple mechanical parts. However, the resulting models suffer from inaccuracies caused by errors in measurement and reconstruction so such models do not have the exact congruence, symmetries and other regularities the original designer intended. We wish to impose such regularities in a beautification process. The paper discusses the particular problem of detecting approximate congruence between parts (e.g. a pair of handles) of a reconstructed B-rep model, so that a subsequent step can enforce them exactly. A practical detection algorithm is given for models defined using planes, spheres, cylinders, cones and tori. Analysis of the algorithm and experimental results show that expected congruence are detected reasonably quickly.

The main purpose of this research is to develop methods for automatic identification of biological specimens in digital photographs and drawings held in a database. Incorporation of taxonomic drawings into a visual indexing system has not been attempted to date. Diatoms are a single cell microscopic algae that provide a particularly suitable case study. Identification of diatoms is a challenging task due to the huge number of the species, blurred boundaries between species, and life cycle related shape changes. A novel model based on principal curves representing the life cycle related shape variation of a number of diatom species has been developed. Our model is suitable for reconstruction purposes, allowing us to produce drawings of a variety of diatom shapes, thus providing a link between the photographs and drawings. We present the classification results of photographed and drawn specimens based on the model and compare our results to another recent system for diatom identification. Finally, given a diatom specimen, we are able not only to identify the species it belongs to but also to pinpoint the stage in the life cycle it represents.

Boundary representation models reconstructed from 3D range data suffer from various inaccuracies caused by noise in the data and the model building software. The quality of such models can be improved in a beautification step, which finds geometric regularities approximately present in the model and tries to impose a consistent subset of these regularities on the model. A framework for beautification and numerical methods to select and solve a consistent set of constraints deduced from a set of regularities are presented. For the initial selection of consistent regularities likely to be part of the model's ideal design priorities, and rules indicating simple inconsistencies between the regularities are employed. By adding regularities consecutively to an equation system and trying to solve it by using quasi-Newton optimization methods, inconsistencies and redundancies are detected. The results of experiments are encouraging and show potential for an expansion of the methods based on degree of freedom analysis.

We present a new method for automatic landmark extraction from the contours of biological specimens. Our ultimate goal is to enable automatic identification of biological specimens in photographs and drawings held in a database. We propose to use active appearance models for visual indexing of both photographs and drawings. Automatic landmark extraction will assist us in building the models. We describe the results of using our method on drawings and photographs of examples of diatoms, and present an active shape model built using automatically extracted data.

Glaucoma is a serious worldwide disease whose treatment can be improved by early detection. As part of a new clinical approach this paper introduces some preliminary studies in the computerised detection of the disease. In particular we consider the problems of registering 3D laser data with a digital image. We introduce a new method based on windowed mutual information and show that it performs better than the standard mutual information technique.

This paper concerns the study of destination choice modelling, more specifically identifying within some area (e.g. a city) the region where a particular store is the most favourable to be visited by individuals. An influence measure is constructed for each individual, which incorporates the modern technique known as Dempster–Shafer theory. Based on the evidence of the shopping destinations of individuals, geographical regions are found for levels of largest belief and plausibility (within Dempster–Shafer theory) for specific stores being the most favourable to visit. Additionally, this method may be used to identify the possible position of new stores, based on regions of most uncertainty or conflict in store choice. A prototype choice modelling system is introduced to enable the series of associated results to be easily visualized and analysed.

Boundary representation models reconstructed from 3D range data suffer from various inaccuracies caused by noise in the data and the model building software. The quality of such models can be improved in a beautification step, which finds regular geometric patterns approximately present in the model and imposes a maximal consistent subset of constraints deduced from these patterns on the model. This paper presents analysis methods seeking geometric patterns defined by similarities. Their specific types are derived from a part survey estimating the frequencies of the patterns in simple mechanical components. The methods seek clusters of similar objects which describe properties of faces, loops, edges and vertices, try to find special values representing the clusters, and seek approximate symmetries of the model. Experiments show that the patterns detected appear to be suitable for the subsequent beautification steps.

This paper addresses a common problem in the segmentation of range images. We present methods for the least-squares fitting of spheres, cylinders, cones, and tori to 3D point data, and their application within a segmentation framework. Least-squares fitting of surfaces other than planes, even of simple geometric type, has rarely been studied. Our main application areas of this research are reverse engineering of solid models from depth-maps and automated 3D inspection where reliable extraction of these surfaces is essential. Our fitting method has the particular advantage of being robust in the presence of geometric degeneracy, i.e., as the principal curvatures of the surfaces being fitted decrease, the results returned naturally become closer and closer to those surfaces of "simpler type", i.e., planes, cylinders, cones, or spheres, which best describe the data. Many other methods diverge because, in such cases, various parameters or their combination become infinite.

We propose a novel hierarchical model of human dynamics for view independent tracking of the human body in monocular video sequences. The model is trained using real data from a collection of people. Kinematics are encoded using Hierarchical Principal Component Analysis, and dynamics are encoded using Hidden Markov Models. The top of the hierarchy contains information about the whole body. The lower levels of the hierarchy contain more detailed information about possible poses of some subpart of the body. When tracking, the lower levels of the hierarchy are shown to improve accuracy. In this article we describe our model and present experiments that show we can recover 3D skeletons from 2D images in a view independent manner, and also track people the system was not trained on.

Presents an implementation of forward-looking infrared (FLIR) and laser radar (LADAR) data simulation for use in developing a multi-sensor data-fusion automated target recognition (ATR) system. Through the use of commercial models and software, we can create a highly detailed scene model, which provides a rich data set for later processing. We embed our own modules within this software to extract data from the scene and then present it to the FLIR and LADAR sensor models. These models produce simulated range and temperature readings for objects within the scene. These data are used to create a variety of images to aid in the visualisation of the data and to test our ATR system. Frames from approach sequences show the LADAR and FLIR data in different formats. The simulated data and subsequent images are accurate and rapid to produce, and provide invaluable data resources for testing an ATR system.

We present new deterministic methods that, given two eigenspace models-each representing a set of n-dimensional observations-will: 1) merge the models to yield a representation of the union of the sets and 2) split one model from another to represent the difference between the sets. As this is done, we accurately keep track of the mean. Here, we give a theoretical derivation of the methods, empirical results relating to the efficiency and accuracy of the techniques, and three general applications, including the construction of Gaussian mixture models that are dynamically updateable.

This paper provides two algorithms; one for adding eigenspaces, another for subtracting them, thus allowing for incremental updating and downdat-ing of data models. Importantly, and unlike previous work, we keep an ac-curate track of the mean of the data, which allows our methods to be used in classification applications. The result of adding eigenspaces, each made from a set of data, is an approximation to that which would obtain were the sets of data taken together. Subtracting eigenspaces yields a result approximating that which would obtain were a subset of data used. Using our algorithms it is possible to perform “arithmetic ” on eigenspaces without reference to the orig-inal data. We illustrate the use of our algorithms in three generic applications, including the dynamic construction of Gaussian mixture models. 1

Eigenspace models are a convenientway to represent sets of observations with widespread applications, including classi#cation. In this paper we describe a new constructive method for incrementally adding observations to an eigenspace model. Our contribution is to explicitly account forachange in origin as well as a change in the number of eigenvectors needed in the basis set. No other method wehave seen considers change of origin, yet both are needed if an eigenspace model is to be used for classi#cation purposes. We empirically compare our incremental method with two alternatives from the literature and show our method is the more useful for classi#cation because it computes the smaller eigenspace model representing the observations. 1 Introduction The contribution of this paper is a method for incrementally computing eigenspace models in the context of using them for classi#cation. Eigenspace models are widely used in computer vision. Applications include: face recognition #8# where...

Many machine vision tasks, e.g. object recognition and object inspection, cannot be performed robustly from a single image. For certain tasks (e.g. 3D object recognition and automated inspection) the availability of multiple views of an object is a requirement. This paper presents a novel approach to selecting a minimised number of views that allow each object face to be adequately viewed according to specified constraints on viewpoints and other features. The planner is generic and can be employed for a wide range of multiple view acquisition systems, ranging from camera systems mounted on the end of a robot arm, i.e. an eye-in-hand camera setup, to a turntable and fixed stereo cameras to allow different views of an object to be obtained. The results (both simulated and real) given focus on planning with a fixed camera and turntable. 1 Introduction Considerable research in machine vision has been directed at object recognition and object inspection. For complete 3D recogn...

This paper deals with the problem of tracking an object from a sequence of images captured from a camera moving towards the object. Many conventional image based trackers trace key reference points over an image sequence using simple correlation processes [1]. However, the presence of noise and the magnification of image features as the camera approaches the object may cause a tracked point to drift. This paper introduces an alternative technique based on tracking regions formed by image segmentation. A region template of the target is acquired in the first frame and correlated over segmented images in future frames. The new technique produces significant reductions in drift rate and results have been obtained for real forward looking infrared (FLIR) images. This paper then discusses the advantages of integrating the new technique with a conventional point correlation based tracker. 1 Introduction The tracking of features in an image sequence is important in the areas of ...

article Interactive hypermedia courseware for the World Wide Web Share on Authors: A. D. Marshall Department of Computing Science, University of Wales, Cardiff, CF2 3XF Department of Computing Science, University of Wales, Cardiff, CF2 3XFView Profile , S. Hurley Department of Computing Science, University of Wales, Cardiff, CF2 3XF Department of Computing Science, University of Wales, Cardiff, CF2 3XFView Profile Authors Info & Claims ACM SIGCUE OutlookVolume 24Issue 1-3Jan.-July, 1996 pp 1–5https://doi.org/10.1145/1013718.237478Online:01 January 1996Publication History 11citation793DownloadsMetricsTotal Citations11Total Downloads793Last 12 Months3Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access