A typical gaming scenario, as developed in the past 20 years, involves a player interacting with a game using a specialized input device, such as a joystic, a mouse, a keyboard, etc. Recent technological advances and new sensors (for example, low cost commodity depth cameras) have enabled the introduction of more elaborated approaches in which the player is now able to interact with the game using his body pose, facial expressions, actions, and even his physiological signals. A new era of games has already started, employing computer vision techniques, brain-computer interfaces systems, haptic and wearable devices. The future lies in games that will be intelligent enough not only to extract the player's commands provided by his speech and gestures but also his behavioral cues, as well as his/her emotional states, and adjust their game plot accordingly in order to ensure more realistic and satisfactory gameplay experience. This special issue on modern control for computer games discusses several interdisciplinary factors that influence a user's input to a game, something directly linked to the gaming experience. These include, but are not limited to, the following: behavioral affective gaming, user satisfaction and perception, motion capture and scene modeling, and complete software frameworks that address several challenges risen in such scenarios.

This paper presents a new database suitable for both 2-D and 3-D face recognition based on photometric stereo (PS): the Photoface database. The database was collected using a custom-made four-source PS device designed to enable data capture with minimal interaction necessary from the subjects. The device, which automatically detects the presence of a subject using ultrasound, was placed at the entrance to a busy workplace and captured 1839 sessions of face images with natural pose and expression. This meant that the acquired data is more realistic for everyday use than existing databases and is, therefore, an invaluable test bed for state-of-the-art recognition algorithms. The paper also presents experiments of various face recognition and verification algorithms using the albedo, surface normals, and recovered depth maps. Finally, we have conducted experiments in order to demonstrate how different methods in the pipeline of PS (i.e., normal field computation and depth map reconstruction) affect recognition and verification performance. These experiments help to 1) demonstrate the usefulness of PS, and our device in particular, for minimal-interaction face recognition, and 2) highlight the optimal reconstruction and recognition algorithms for use with natural-expression PS data. The database can be downloaded from http://www.uwe.ac.uk/research/Photoface.

It is argued that robotic platforms are the way forward towards building intelligent systems, where multiple sensors and manipulation are used for cognitive processes. It is also argued that the cue for developing the right architecture for such a system is human language.

This letter presents a novel probabilistic framework for augmenting the recognition performance of biometric systems with information from continuous soft biometric (SB) traits. In particular, by modelling the systematic error induced by the estimation of the SB traits, a modified efficient recognition probability can be extracted including information related both to the hard and SB traits. The proposed approach is applied without loss of generality in the case of gait recognition, where two state-of-the-art gait recognition systems are considered as hard biometrics and the height and stride length of the individuals are considered as SBs. Experimental validation on two known, large datasets illustrates significant advances in the recognition performance with respect to both identification and authentication rates.

The eight papers in this special section focus on space technology.

We propose a robust approach to discriminant kernel-based feature extraction for face recognition and verification. We show, for the first time, how to perform the eigen analysis of the within-class scatter matrix directly in the feature space. This eigen analysis provides the eigenspectrum of its range space and the corresponding eigenvectors as well as the eigenvectors spanning its null space. Based on our analysis, we propose a kernel discriminant analysis (KDA) which combines eigenspectrum regularization with a feature-level scheme (ER-KDA). Finally, we combine the proposed ER-KDA with a nonlinear robust kernel particularly suitable for face recognition/verification applications which require robustness against outliers caused by occlusions and illumination changes. We applied the proposed framework to several popular databases (Yale, AR, XM2VTS) and achieved state-of-the-art performance for most of our experiments.

This paper presents an effective registration method between two geophysical images from different modalities, namely magnetic and electrical, in order to increase the information obtained from the two original images independently. In order to achieve the registration, random local transformations were applied in 5×5 randomly chosen windows of the sensed image while the mutual information was used as the similarity measure. The results, in all cases, show a significant increase in mutual information in comparison with the registration through geographical coordinates.

Vector field tomography is a field that has received considerable attention in recent decades. It deals with the problem of the determination of a vector field from non-invasive integral data. These data are modelled by the vectorial Radon transform. Previous attempts at solving this reconstruction problem showed that tomographic data alone are insufficient for determining a 2D band-limited vector field completely and uniquely. This paper describes a method that allows one to recover both components of a 2D vector field based only on integral data, by solving a system of linear equations. We carry out the analysis in the digital domain and we take advantage of the redundancy in the projection data, since these may be viewed as weighted sums of the local vector field's Cartesian components. The potential of the introduced method is demonstrated by presenting examples of vector field reconstruction.

In this paper, we propose a novel logic-rule learning approach for the Tower of Knowledge (ToK) architecture, based on Markov logic networks, for scene interpretation. This approach is in the spirit of the recently proposed Markov logic networks for machine learning. Its purpose is to learn the soft-constraint logic rules for labeling the components of a scene. In our approach, FOIL (First Order Inductive Learner) is applied to learn the logic rules for MLN and then gradient ascent search is utilized to compute weights attached to each rule for softening the rules. This approach also benefits from the architecture of ToK, in reasoning whether a component in a scene has the right characteristics in order to fulfil the functions a label implies, from the logic point of view. One significant advantage of the proposed approach, rather than the previous versions of ToK, is its automatic logic learning capability such that the manual insertion of logic rules is not necessary. Experiments of labeling the identified components in buildings, for building scene interpretation, illustrate the promise of this approach.

Endmember extraction is usually based on the solution of a system of linear equations that allows the identification of some basic spectra in terms of which the observed mixed spectra may be expressed as linear combinations. In this paper, we propose to close the loop of such an approach by identifying only the basic spectra that reproduce the dominant cover classes of a region as endmembers, and distinguishing them from outlier spectra present in the scene. The latter are often confused by other methods as endmember spectra, whereas in many practical applications, they are treated as anomalies or targets in the scene. Thus, the proposed method identifies endmembers in a robust way, separating them from outliers.

In this paper, we present a new method for bidirectional relighting for 3D-aided 2D face recognition under large pose and illumination changes. During subject enrollment, we build subject-specific 3D annotated models by using the subjects' raw 3D data and 2D texture. During authentication, the probe 2D images are projected onto a normalized image space using the subject-specific 3D model in the gallery. Then, a bidirectional relighting algorithm and two similarity metrics (a view-dependent complex wavelet structural similarity and a global similarity) are employed to compare the gallery and probe. We tested our algorithms on the UHDB11 and UHDB12 databases that contain 3D data with probe images under large lighting and pose variations. The experimental results show the robustness of our approach in recognizing faces in difficult situations.

This paper addresses the problem of identifying the components (such as balconies and windows) of the 3D model of a building. A novel method, based on a voting scheme, is presented for solving such a problem. It is intuitive that interference (such as shadows and occlusions) rarely happen at the same place or at different times when looking at a scene from different directions. In the spirit of this intuition, the voting-based method combines the information from various images to identify and segment the components of a building.

We present the results of an experiment designed to investigate the way information is organized and stored in the human brain. In particular, we are using controlled stimuli to reverse engineer the networks of ideas and concepts in order to answer the following questions. (1) Are the networks of ideas and concepts in the human brain invoked by verbal and visual stimuli distinct from each other? The answer appears to be no for the network of ideas and inconclusive for the network of concepts. (2) What is the topology of these networks? Our experimental results show that both are small-world networks, with the network of ideas being random and the network of concepts scale-free.

In this paper, general solutions for nonlinear non-negative component analysis for data representation and recognition are proposed. Motivated by a combination of the non-negative matrix factorization (NMF) algorithm and kernel theory, which has lead to a recently proposed NMF algorithm in a polynomial feature space, we propose a general framework where one can build a nonlinear non-negative component analysis method using kernels, the so-called projected gradient kernel non-negative matrix factorization (PGKNMF). In the proposed approach, arbitrary positive definite kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem. Moreover, we propose fixed point algorithms for the special case of Gaussian radial basis function (RBF) kernels. We demonstrate the power of the proposed methods in face and facial expression recognition applications.

In this paper general solutions for nonlinear nonnegative component analysis for data representation and recognition are proposed. That is, motivated by a combination of the Nonnegative Matrix Factorization (NMF) algorithm and kernel theory, which has lead to an NMF algorithm in a polynomial feature space, we propose a general framework where one can build a nonlinear nonnegative component analysis using kernels, the so-called projected gradient Kernel Nonnegative Matrix Factorization (PGKNMF). In the proposed approach, arbitrary positive kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem. Moreover, we propose fixed point algorithms for the special case of Radial Basis Function (RBF) kernels. We demonstrate the power of the proposed methods in face and facial expression recognition applications.

Cognitive systems are trained to recognise perceptually meaningful parts of an image. These regions contain some variation, i.e. local texture, and are roughly convex. We call such regions ¿blobs¿. We define blobs to be components that merit further analysis by a higher level interpretation module as they very likely constitute semantically meaningful units, rather than characteristic features or salient spots. A scheme, independent of scale and colour, is proposed, based on the use of Gaussian kernels and mathematical morphology for the extraction of blobs. For understanding how well the extracted blobs match the meaningful regions, we present an eye-tracking experiment using 20 subjects and 20 different colour images using the hypothesis that the gaze of the viewers are more attracted to the meaningful regions/objects of a scene. We show that the gaze of the subjects is attracted more to the regions which were extracted by our model in comparison with the regions which were extracted by the saliency map model, proposed by Itti et al.

This paper proposes a novel tracking methodology for mul-tispectral and hyperspectral sequences. Our approach combines techniques used in multispectral and hyperspectral anomaly detection with a Kalman-Filter (KF) for tracking. The algorithm takes advantage of the additional information provided by the spectra in multispectral and hyperspectral sequences and combines it with a KF to track a target in the presence of occlusions. The proposed algorithm is based on modeling the tracked object's local background with the help of a Self Organizing Map (SOM), followed by the construction of a 2D Centre of Gravity Map (CoGM), the entries of which lead to the localisation of the target's current position. The KF, in prediction mode, is employed in order to perform robust tracking during occlusion.

Most available methods for endmember extraction use the convexity of the data structure and consider the vertices of the data as the purest pixels. Such methods do not consider the applicability of the linear mixing model once the endmembers have been extracted. Thus they might return false endmembers if the data contain outliers such as anomalies. In this paper we tackle this problem by identifying endmembers in a robust way, separating them from outliers. We tested the proposed algorithm with real and synthetic data and compared it with the VCA, SGA and N-FINDR algorithms, showing better and more robust endmember extraction.

We propose a low false alarm methodology to determine anomalies in hyperspectral data. The method is based on the assumptions that the linear mixing model is valid and that, due to the resolution of the image, most pixels are mixtures of common substances, of which pure pixels (not mixtures) are rare. In the first stage of the algorithm, the classes associated with the background, which are the dominant classes in the image, are found by clustering the image pixels. The resulting clusters may be considered as representatives of the background classes in the image. In order to determine the anomalous pixels, a threshold may be applied to the distance between the pixel spectrum and the cluster centers. However, pixels corresponding to anomalies and pure substances will both show high distances. If we consider that the background classes are themselves most likely mixtures of other materials, the pixels within the convex hull formed by the background classes will have positive fractions that are smaller than one. The pure substances, however, will be outside such a convex hull and will show negative or superunity fractions. Pixels with such mixing proportions are explained as linear combinations of the background classes and, therefore, as not true anomalies. Pixels corresponding to anomalies, however, when expressed as linear combinations of the background classes, show high residual error even with negative and superunity mixing proportions. We use the unmixing spectral linear model without the nonnegativity constraint to distinguish between false anomalies corresponding to pure substances and real man-made anomalies.

We propose a methodology with which one can model the interactions of agents in a digital business ecosystem. We assume that the system is closed and static and define a global cost function that may be used to identify the states of the system. The cost function is minimized with the help of the Langevin equation. We demonstrate the use of the proposed model with the help of simulated data.

We present a generalisation of the 4-light photometric stereo technique to an arbitrary number Q of illuminants. We assume that the surface reflectance can be approximated by the Lambertian model plus a reflectance component. The algorithm works in a recursive manner eliminating the pixel intensities affected by shadows or highlights based on a least squares error technique, retaining only the information coming from illumination directions that can be used for photometric stereo reconstruction of the normal of the corresponding surface patch. We report results for both simulated and real surfaces and compare them with the results of other state of the art photometric stereo algorithms. 1

We present in this paper an algorithm inspired by the human visual system to reconstruct images from irregularly placed samples. This algorithm is based on linear spline approximations with control points on a hexagonal grid. Several spline approximations are computed for different transformations of the control point grid (e.g. translations and rotations). These approximations are then merged together after compensation of the transformations, yielding a high-quality invariant image reconstruction. Evaluations show that the use of hexagonal grids and of the “invariance by integration ” principle improves reconstruction quality. 1

The Tower of Knowledge architecture integrates probability theory and logic for making decisions. The scheme models the causal dependencies between the functionalities of objects and their descriptions, and then employs the maximum expected utility principle, which combines probability theory and logic, to select the most appropriate label for the object. Since most existing scene interpretation methods rely heavily on training data, we develop in this paper a recursive version of ToK to avoid such dependency. Recursive ToK learns the prior distributions iteratively from the decisions of labelling components made in the last iteration, partly by functionalities of components, and partly by the already learnt prior distributions in previous iterations. To validate our method in the domain of 3D outdoor scene interpretation, we compare ToK against a state-of-the-art method, Expandable Bayesian Networks (EBN), for labelling components of buildings. Experimental results then show that the labelling accuracy of ToK is superior to that of EBN. Also, these results reveal that recursive ToK improves the accuracy of ToK for labelling 3D components in the worst case when lacking any training data. 1

We present a recursive algorithm for 3D surface reconstruction based on photometric stereo in the presence of highlights, and self and cast shadows. We assume that the surface reflectance outside the highlights can be approximated by the Lambertian model. The algorithm works with as few as three light sources, and it can be generalised for N without any difficulties. Furthermore, this reconstruction method is able to identify areas where the majority of the lighting directions result in unreliable pixel intensities, providing the capability to adjust a reconstruction algorithm and improve its performance avoiding the unreliable sources. We report results for both artificial and real images and compare them with the results of other state of the art photometric stereo algorithms.

This paper presents a fuzzy expert system for the creation of landslide possibility maps using change of land-use data from Earth observation, as well as historical, rainfall, and earthquake data stored in a geographic information system, as input. The difference with other systems is in the use of change (differential) input data. The method is tested with 16 documented landslides. The fuzzy neural network (NN) developed can predict the crowns of 13 out of the 16 landslides to be among the 5% most at-risk pixels that are identified in the area of study, which covers 100 km2. The fuzzy expert system considers the rules that increase the possibility of a landslide, as supplied by experts, and expresses them in the form of an empirical algebraic formula. It then fuzzifies the various thresholds they rely on and, in conjunction with uncertainties that are reported by the classifier that decides the land-use change, produces a fuzzy algebraic formula that may be used to identify the range of uncertainty in the possibility of a landslide in terms of the ranges of uncertainty in the input variables. This formula is used to train an Ishibuchi fuzzy NN, which has been designed to capture uncertainty in the rules and uncertainty in the input variables. It is this Ishibuchi NN that acts as a fuzzy expert system.

In this paper we propose a non-Gibbsian Markov random field to model the spatial and topological relationships between objects in structured scenes. The field is formulated in terms of conditional probabilities learned from a set of training images. A locally consistent labelling of new scenes is achieved by relaxing the Markov random field directly using these conditional probabilities. We evaluate our model on a varied collection of several hundred handsegmented images of buildings. 1

We investigate new ways of improving the performance of Bayesian-based super-resolution image reconstruction by using a discontinuity adaptive image prior distribution based on robust statistics and a fast and efficient way of initializing the optimization process. The latter is an adapted normalized convolution (NC) technique that incorporates the uncertainty induced by registration errors. We present both qualitative and quantitative results on real video sequences and demonstrate the advantages of the proposed method compared to conventional methodologies.

In this work we address the problem of land cover classification in advanced synthetic aperture radar (ASAR) images. The derivation and assessment of texture features for ASAR image segmentation is investigated using full multidimensional co-occurrence matrices as features. Expansion of local patches in terms of Walsh functions helps identify the optimal distance for the calculation of the co-occurrence matrices. The defined distance agrees with the one chosen by performing exhaustive tests where many distances were tried and the best was chosen from the training data. The well known chi-square test of statistical significance has been used for classification.

We propose a low false alarm methodology to determine anomalies in hyperspectral data. The method is based on the assumption that due to the resolution of the image, most pixels are mixtures of pure substances, which are relatively rare in the scenes. In the first stage of the algorithm, the background classes, which are the dominant classes in the image, are found. The method consists of robust clustering of a randomly–chosen small percentage of the image pixels. The resulting clusters may be considered as representatives of the background classes in the image. In order to determine the anomalous pixels, a threshold may be applied to the distance between the pixel spectrum and the cluster centres. However, pixels corresponding to anomalies and pure substances will both show high distances. If we consider that the background classes are themselves most likely mixtures of other materials, the pixels within the convex hull formed by the background classes will have positive fractions that are smaller than 1. The pure substances, however, will be outside such a convex hull, and will show negative or superunity fractions. We propose to use the unmixing spectral linear model without the non–negativity constraint, to distinguish between false anomalies corresponding to pure substances and real man–made anomalies.

The purpose of the Applied Multi-dimensional Fusion Project is to investigate the benefits that data fusion and related techniques may bring to future military Intelligence Surveillance Target Acquisition and Reconnaissance systems. In the course of this work, it is intended to show the practical application of some of the best multi-dimensional fusion research in the UK. This paper highlights the work done in the area of multi-spectral synthetic data generation, super-resolution, joint fusion and blind image restoration, multi-resolution target detection and identification and assessment measures for fusion. The paper also delves into the future aspirations of the work to look further at the use of hyper-spectral data and hyper-spectral fusion. The paper presents a wide work base in multi-dimensional fusion that is brought together through the use of common synthetic data, posing real-life problems faced in the theatre of war. Work done to date has produced practical pertinent research products with direct applicability to the problems posed.

We propose a computationally efficient method for determining anomalies in hyperspectral data. In the first stage of the algorithm, the background classes, which are the dominant classes in the image, are found. The method consists of robust clustering of a randomly chosen small percentage of the image pixels. The clusters are the representatives of the background classes. By using a subset of the pixels instead of the whole image, the computation is sped up, and the probability of including outliers in the background model is reduced. Anomalous pixels are the pixels with spectra that have large relative distances from the cluster centers. Several clustering techniques are investigated, and experimental results using realistic hyperspectral data are presented. A self-organizing map clustered using the local minima of the U-matrix (unified distance matrix) is identified as the most reliable method for background class extraction. The proposed algorithm for anomaly detection is evaluated using realistic hyperspectral data, is compared with a state-of-the-art anomaly detection algorithm, and is shown to perform significantly better.

This paper presents a rule-based expert system used to create landslide warning and alert maps using as input change of landuse data from Earth observation, as well as geological, rainfall, and earthquake data stored in a Geographic Information System. The difference with other systems is in the use of change (differential) input data. The system is tested with 16 documented landslides. It can predict 11 of the 16 landslides to be among the 10% most at risk pixels in the area of study, which covers 100 km2

A target material may be considered as an anomaly in an image, having different spectral signatures from the spectral signatures of the background objects. In order to detect such anomalies in an image, the classes associated with the background have to be known. A computationally efficient methodology to determine the background pure classes present in a low resolution hyperspectral image has been previously proposed by the authors. The method is based on robust clustering using a small percentage of the image pixels as input. The clusters are obtained using a self organising map (SOM) clustered using the local minima of the U-matrix (distance matrix). In this paper, we provide a statistical study and evaluation of the proposed approach using simulated and real hyperspectral images. In particular, we answer the question:"what sampling rate should I use in order to be x% confident that I picked up y% of the background classes?".

In this paper, we assume that we are given the images of two segmented objects, one of which may be an affinely distorted version of the other, and wish to recover the values of the parameters of the affine transformation between the two images. The images may also differ by the overall level of illumination. The multiplicative constant of such difference may also be recovered. We present a generic theoretical framework to solve this problem. In terms of this framework, other proposed methods may be interpreted. We show how, in this framework, one can recover the affine parameters in a way that is robust to various effects, such as occlusion and illumination variation. The proposed method is generic enough to be applicable also to matching two images that do not depict the same scene or object.

In this paper, we present an algorithm that allows the simultaneous calculation of several cross correlations. The algorithm works by shifting the range of values of different images/signals to occupy different orders of magnitude and then combining them to form a single composite image/signal. Because additional signals are placed in the space usually occupied by a single signal, we call this the "invaders algorithm," to imply that extra signals invade the space that normally belongs to a single signal. After correlation is performed, the individual results are recovered by performing the inverse operation. The limitations of the algorithm are imposed by the finite length of the mantissa of the hardware used, the precision of the algorithm that performs the cross correlation (e.g., the precision of the fast Fourier transform (FFT)) and by the actual values of the images/signals that are to be combined. The algorithm does not require any special hardware or special FFT algorithm. For typical 256 x 256 images, an acceleration by a factor of at least two in the calculation of their cross correlations is guaranteed using an ordinary PC or a laptop. As for smaller sized templates, tenfold accelerations may be achieved.

This paper presents a new algorithm which can be used to register images of the same or different modalities, e.g., images with multiple channels, such as X-rays, temperature or elevation, or simply images of different spectral bands. In particular, a correlation-based scheme is used, but instead of gray values, it correlates numbers formulated by different combinations of the extracted local Walsh coefficients of the images. Each image patch is expanded in terms of Walsh basis functions. Each Walsh basis function can be thought of as measuring a different aspect of local structure, e.g., horizontal edge, corner, etc. The coefficients of the expansion, therefore, can be thought of as dense local features, estimating at each point the degree of presence of, for example, a horizontal edge, a corner with contrast of a certain type, etc. These coefficients are normalized and used as digits in a chosen number system which allows one to create a unique number for each type of local structure. The choice of the basis of the number system allows one to give different emphasis to different types of local feature (e.g., corners versus edges), and, thus, the method we present forms a unified framework in terms of which several feature matching methods may be interpreted. The algorithm is compared with wavelet and contour based approaches, using simulated and real images. The two images are assumed to differ from each other by a rotation and a translation only.

The human vision system can discriminate regions which differ up to the second-order statistics only. We present an algorithm designed to reveal "hidden" boundaries in gray level images, by computing gradients in higher order statistics of the data. We demonstrate it by applying it to the identification of possible "hidden" boundaries of glioblastomas as manifest themselves in three-dimensional (3-D) MRI scans, using a model driven approach. We also demonstrate the method using a nonmodel driven approach where we have no prior information about the location of possible boundaries. In this case, we use 3-D MRI data concerning schizophrenic patients and normal controls.

In order to detect a target or anomaly in a hyper-spectral image the classes associated with the background have to be identified. We propose a computationally efficient methodology to determine the background classes present in the image. The method is based on the assumption that mixed and anomaly pixels are relatively rare in comparison with the abundance of the background class pixels. The method considers the background classes as groups of distinct measurements and consists of robust clustering of a randomly picked small percentage of the image pixels. The resulting clusters may be considered as representatives of the background of the image. Several clustering techniques are investigated and experimental results using hyperspectral data are presented. The proposed technique using a self-organising map is then compared with a state-of the art endmember extraction technique.

Super-resolution algorithms produce a single high-resolution image from a set of several, low-resolution images of the desired scene. The low-resolution frames are shifted differently with respect to the high resolution frame with subpixel increments. This paper presents first a theoretical overview of super-resolution algorithms. The most important methods, namely, the iterative back-projection, projection onto convex sets, and maximum a posteriori estimation are then compared within the same framework of implementation.

We propose a method for extracting automatically and simultaneously the quasi-horizontal surfaces in three-dimensional (3-D) seismic data. The proposed algorithm identifies connected sets of points which form surfaces in 3-D space. To improve reliability, this algorithm takes into consideration the relative positions of all horizons, and uses globally self-consistent connectivity criteria which respect the temporal order of horizon creation. The first stage of the algorithm consists of the preliminary estimation of the local direction of each horizon at each point of the 3-D space. The second stage consists of smoothing the signal along the detected layer structure to reduce noise. The last stage consists of the simultaneous building of all 3-D horizons. The output of the processing is a set of 3-D horizons represented by a series of triangulated surfaces.

In this paper we investigate the effect of the illuminant tilt rotation over surface textures by analysing a set of image texture features extracted from the co-occurrence matrix. From the behaviour of each texture feature, a simple method able to predict the illuminant tilt angle of test images is developed. Moreover, the method is also used to perform a texture classification invariant to the illuminant tilt angle rotation. This study and experimental results over different real textures show that the illumination tilt angle can be accurately predicted, as part of the texture classification process. 1

We introduce a face representation, the shape trace transform (STT), for recognizing faces in an authentication system. The STT offers an alternative representation for faces that has a very high discriminatory power. We estimate the dissimilarity between two shapes by a new measure. We propose the Hausdorff context. Reinforcement learning is used to search the optimal parameters of the algorithm, for which the within class variance of the STT is minimized. This research demonstrates that the proposed method provides a new way for face representation. Our system is verified with experiments on the XM2VTS database.

We present a very fast algorithm for image registration, which reduces the computational cost significantly in comparison with the direct approach of trial and error. The proposed approach is appropriate for distortions which form a group, ie the transformation to which an image is subject can be expressed as the combination of two other transformations of the same type. We demonstrate it here for the case of shifting, rotation and scaling.

Changes in the angle of illumination incident upon a 3D surface texture can significantly alter its appearance, implying variations in the image texture. These texture variations produce displacements of class members in the feature space, increasing the failure rates of texture classifiers. To avoid this problem, a model-based texture recognition system which classifies textures seen from different distances and under different illumination directions is presented in this paper. The system works on the basis of a surface model obtained by means of 4-source colour photometric stereo, used to generate 2D image textures under different illumination directions. The recognition system combines coocurrence matrices for feature extraction with a Nearest Neighbour classifier. Moreover, the recognition allows one to guess the approximate direction of the illumination used to capture the test image.

We present an algorithm for separating the local gradient information and Lambertian color by using 4-source color photometric stereo in the presence of highlights and shadows. We assume that the surface reflectance can be approximated by the sum of a Lambertian and a specular component. The conventional photometric method is generalized for color images. Shadows and highlights in the input images are detected using either spectral or directional cues and excluded from the recovery process, thus giving more reliable estimates of local surface parameters.

The appearance of a rough surface is affected by the direction from which it is lit and texture classifiers should account for this. We propose a classifier that is robust to lighting direction—even when the direction is unknown. An existing model of the dependency of texture features on lighting direction is used to develop a probabilistic model. Given a feature set, the algorithm estimates the most likely illumination direction for each texture class. The likelihoods of each candidate (with their estimated lighting) are compared to classify the sample. The ability of the classifier to identify illuminant direc-tion, and to assign the correct class, was tested on 25 real texture samples. The classifier was able to accurately estimate both the azimuth and the zenith of the light source for most textures and gave a 98 % classification rate. 1

The Trace transform is a generalisation of the Radon transform that allows one to construct image features that do not necessarily have meaning in terms of human perception, but they measure different image characteristics. The ability of producing thousands of features from an image allows one to be selective as to which are appropriate for a particular task. In this paper we propose the use of such an approach to the problem of texture discrimination and compare its results with the classical co-occurrence matrix approach where usually the features used are fewer than ten. 1

The Trace transform is a generalisation of the Radon transform that allows us to construct image features that are invariant to a chosen group of image transformations. In this paper we propose functionals which can be computed from the image function, and which can be used to describe objects in a way that is invariant to the group of affine transforms. We demonstrate the usefulness of the constructed image descriptors in retrieving images from an image database and compare it with the moments based method.

Classification of a rough 3D surface from 2D images may be difficult due to directional effects introduced by illumination. One possible way of dealing with the problem is to extract the local albedo and gradient surface information which do not depend on the illumination, and classify the texture directly using these intrinsic characteristics. In this paper we present an algorithm for simultaneous recovery of local gradient and colour using multiple photometric images. The algorithm is proven to be optimal in the least squares error sense. Experimental results with real images and comparison with other approaches are also presented.

The photometric stereo technique (PS) has been used for shape recovery for quite some time now. Using a modification of the method, we can recover local surface normals and surface colour from multiple images for surfaces with unspecified reflectance function. However, for adequate recovery one should be able to detect and appropriately deal with shadows and highlights in the input images. We propose here a methodology that allows us to do that.

We propose functionals which can be computed from the image function, and which can be used to derive features that are invariant to the group of affine transforms. The first step of this computation is the computation of the trace transform of the image. The trace transform is a generalisation of the Radon transform, which computes a functional, not necessarily the integral, along lines tracing the image. Two further functionals computed from the trace transform result in an image feature. The choice of these functionals determines the type of invariance of this feature.

The Trace transform proposed, a generalization of the Radon transform, consists of tracing an image with straight lines along which certain functionals of the image function are calculated. Different functionals that can be used may be invariant to different transformations of the image. The paper presents the properties the functionals must have in order to be useful in three different applications of the method: construction of invariant features to rotation, translation and scaling of the image, construction of sensitive features to the parameters of rotation, translation and scaling of the image, and construction of features that may correlate well with a certain phenomenon we wish to monitor.

Error models of slope and aspect of a terrain are presented. Such data are often extracted from a geographic information system (GIS), which may contain information from digital maps and remote sensing images. Although the sources of these data are usually of diverse resolution, all of them are usually resampled to refer to the same resolution. The authors examine the error associated with such data because of subsampling. The error distributions are modeled empirically.

The authors propose some statistics of distributions of sets of pixels corresponding to rough surfaces, which are illumination invariant and therefore they can characterize the distributions irrespective of the solar angle. The illumination invariant statistics are used to solve the linear spectral unmixing problem for sets of mixed pixels, taking into consideration the distortion introduced to the statistics due to surface texture. The spectral unmixing of sets of mixed pixels is formulated in terms of the invariant statistics and the method is demonstrated using simulated data.

The wide usage of small satellite imagery, especially its commercialization, makes data-based onboard compression not only meaningful but also necessary in order to solve the bottleneck between the huge volume of data generated onboard and the very limited downlink bandwidth. The authors propose a method that encodes different regions with different algorithms. The authors use three shape-adaptive image compression algorithms as the candidates. The first one is a JPEG-based algorithm, the second one is based on the object-based wavelet transform method proposed by Katata et al. (1997), and the third adopts Hilbert scanning of the regions of interest followed by one-dimensional (1-D) wavelet transform. The three algorithms are also applied to the full image so that one can compare their performance on a whole rectangular image. The authors use eight Landsat TM multispectral images and another 12 small satellite single-band images as their data set. The results show that these compression algorithms have significantly different performance for different regions.

Feature-based texture segmentation methods often compute the texture features over a window of finite support converting raw texture descriptors into usable texture features. However, this process has the adverse effect of blurring the texture feature boundaries such that features at pixels close to the boundaries are a mixture of raw descriptors from two distributions. We propose a method which gives the least-squares estimate of the proportional mixture of a pixel feature from the two distributions representing the regions on each side of the boundary. In this manner each pixel may be relabelled according to the region distribution which contributes most to that pixel, thus refining the region boundaries.

Most of texture classification techniques are evaluated using large rectangular samples of each texture. This, however, is unrealistic, especially as samples of a certain texture may be used as cues in searching image databases or identifying objects in a scene. In this paper, four different texture classification methods (wavelets, co-occurrence matrices, sum and difference histograms, and 1D Boolean models) are systematically compared and evaluated with respect to their performance in identifying textures from small and irregular samples.

This paper describes an approach to perceptual segmentation of color image textures. A multiscale representation of the texture image, generated by a multiband smoothing algorithm based on human psychophysical measurements of color appearance is used as the input. Initial segmentation is achieved by applying a clustering algorithm to the image at the coarsest level of smoothing. The segmented clusters are then restructured in order to isolate core clusters, i.e., patches in which the pixels are definitely associated with the same region. The image pixels representing the core clusters are used to form 3D color histograms which are then used for probabilistic assignment of all other pixels to the core clusters to form larger clusters and categorise the rest of the image. The process of setting up color histograms and probabilistic reassignment of the pixels to the clusters is then propagated through finer levels of smoothing until a full segmentation is achieved at the highest level of resolution.

The authors propose an improved version of JPEG coding for compressing remote sensing images obtained by optical sensors onboard microsatellites. The approach involves expanding cloud features to include their cloud-land transitions, thereby simplifying their coding and subsequent compression. The system is fully automatic and appropriate for onboard implementation. Its improvement in coding stems from the realization that a large number of bits are used for coding the blocks that contain the transition regions between bright clouds, if present in the image, and the dark background. A fully automatic cloud-segmentation algorithm is therefore used to identify the external boundaries of the clouds, then smooth the corresponding blocks prior to coding. Further gains are also achieved by modifying the quantization table used for coding the coefficients of the discrete cosine transform. Compared to standard JPEG, at the same level of reconstruction quality, the new method can achieve compression ratio improvement by 13-161%, depending upon the context and the amount of cloud present in the specific image. The results are demonstrated with the help of several real images obtained by the University of Surrey, U.K., satellites.

In this paper, we develop the theory of probabilistic relaxation when the objects to be labeled are arranged in a rectangular grid with known adjacency relations. In this case a dictionary of permissible label configurations is available. The novelty of this work lies in the inclusion of measurements concerning binary relations between the objects to be labeled. These are compared with the corresponding binary relations between the nodes of the dictionary. This way, one of the major objections to probabilistic relaxation, namely, the disregard of the data after the initial assignment of probabilities, is removed. The theory we develop is demonstrated by applying it to the problem of edge relaxation labeling. We show that the inclusion of binary relations greatly improves the performance of algorithms of this kind and compare our approach with previously developed dictionary based approaches, both theoretically and experimentally. Also, a comparison with other edge-postprocessing strategies is provided.

The authors propose a method that estimates the membership of a mixed pixel in various possible mixture proportions. The method relies on the creation of model mixtures and the estimation of their local density in the vicinity of the pixel under consideration.

The one-dimensional (1-D) Boolean model is used to calculate features for the description of binary textures. Each two-dimensional (2-D) texture is converted into several 1-D strings by scanning it according to raster vertical, horizontal or Hilbert sequences. Several different probability distributions for the segment lengths created this way are used to model their distribution. Therefore, each texture is described by a set of Boolean models. Classification is performed by calculating the overlapping probability between corresponding models. The method is evaluated with the help of 32 different binary textures, and the pros and cons of the approach are discussed.

Two approaches to the characterization of three-dimensional (3-D) textures are presented: one based on gradient vectors and one on generalized co-occurrence matrices. They are investigated with the help of simulated data for their behavior in the presence of noise and for various values of the parameters they depend on. They are also applied to several medical volume images characterized by the presence of microtextures and their potential as diagnostic tools and tools for quantifying and monitoring the progress of various pathologies is discussed. No firm medical conclusions can be drawn as not enough clinical data are available. The gradient based method appears to be more appropriate for the characterization of microtextures. It also shows more consistent behavior as a descriptor of pathologies than the generalized co-occurrence matrix approach.

The classification of sets of mixed pixels can be accomplished by making use of the relationship of higher order moments of the distributions of the pure and mixed classes. As a consequence, the number of equations relat-ing the means of the distributions can be augmented, providing a number of linear equations larger than the number of available sensor bands. Thus, the important advantage the method offers and makes it unique is the fact that more classes than the available bands can be identified. The capabilities and limitations of the method are assessed first by the use of simulated data that closely imitate real data, and also by real data from Landsat images. 1

We generalise the use of the 1D Boolean model for the analysis of grey textures. Each grey image is first histogram-equalised with 8 grey levels. The pixels that constitute each grey level are used to form a binary image. Each of these binary images is separately analysed with the help of the 1D Boolean model and features are extracted from it. The final texture recognition is performed on the basis of these features. 100% recognition rate is achieved for an image database of Brodatz textures that consists of 48 model textures coming from 16 different classes and 16 test textures, one from each class.

In this paper we propose two improvements to standard JPEG coding that can improve the compression ratio achieved for compressing remote sensing images obtained by sensors on-board microsatellites by more than 39-60%. The first improvement consists in using a quantisation table that has been shown experimentally to be more appropriate for remote sensing images. The second and more significant improvement comes from the use of a novel region growing algorithm that can identify the outer border of a cloud region. The blocks that correspond to cloud regions are subsequently smoothed, as they represent unwanted information for the applications we are interested in, and encoded. The results are demonstrated with the help of several real images obtained by the Surrey University satellites.

The trace transform, a generalisation of the Radon transform, consists of tracing an image with straight lines along which certain functionals of the image function are calculated. The paper presents the properties a functional must have in order to allow the estimation of the angle of rotation, shift vector and scaling factor between two images, so that the two images can be brought to complete registration for pixel by pixel comparison for fault detection.

The trace transform proposed, a generalisation of the Radon transform, consists of tracing an image with straight lines along which certain functionals of the image function are calculated. Different functionals that can be used may be invariant to different transformations of the image. The paper presents the properties a functional must have in order to be invariant to rotation, translation and scaling of the image. With the help of these features we can identify, for example, objects depicted in an image database.

In this paper we show how a Bayesian network of inference can be used with a GIS to combine information from different sources of data for classification. Data may include satellite sensor images, topographic maps, geological maps etc, each one with its own resolution and accuracy. We show how this uncertainty in the input data can be incorporated in the network and present various methods by which the conditional probability matrices used by the network can be constructed. We demonstrate our approach within the framework of the problem of assessing the risk of desertification of some burned forests in the Mediterranean region.

We present in this paper a novel method for eliciting the conditional probability matrices needed for a Bayesian network with the help of a neural network. We demonstrate how we can obtain a correspondence between the two networks by deriving a closed-form solution so that the outputs of the two networks are similar in the least square error sense, not only when determining the discriminant function, but for the full range of their outputs. For this purpose we take into consideration the probability density functions of the independent variables of the problem when we compute the least square error approximation. Our methodoloy is demonstrated with the help of some real data concerning the problem of risk of desertification assessment for some burned forests in Attica, Greece where the parameters of the Bayesian network constructed for this task are successfully estimated given a neural network trained with a set of data.

This paper presents the application of a very efficient multiresolution transformation, which is related to the renormalization group approach of physics, to the problem of motion segmentation. The approach proposed is much faster and yields much better results than the full resolution approach. The problem is formulated as one of global optimization where a cost function is constructed to combine the information obtained by various processors as well as the constraints we impose to the problem. The cost function is optimized using the supercoupling multiresolution approach.

We argue that for a method to be useful in practice, it is necessary to perform its sensitivity analysis and to investigate its robustness to measurement errors. We present here a complete sensitivity analysis of the 3D reconstruction method based on projective geometry. We use this sensitivity analysis to best design a system for the inference of the shape of a block of granite from cameras placed at 90 degrees angular separation. The system has been tested on both real and synthetic data.

In this work, we present a segmentation algorithm for color images that uses the watershed algorithm to segment either the two-dimensional (2-D) or the three-dimensional (3-D) color histogram of an image. For compliance with the way humans perceive color, this segmentation has to take place in a perceptually uniform color space like the Luv space. To avoid oversegmentation, the watershed algorithm has to be applied to a smoothed histogram.

The authors present a novel method for mixed pixel classification where the classification of groups of mixed pixels is achieved by taking into consideration the higher order moments of the distributions of the pure and the mixed classes. The equations expressing the relationship between the higher order moments are used to augment the set of equations that express the relationship between the means only. The authors show that weighting these equations does not make the set of equations available less reliable. As a consequence, the number of equations can be increased and thus more classes than available bands can be identified. The method is exhaustively tested using simulated data and is also applied to real Landsat TM data for which ground data are available.

The authors present a novel method for mixed pixel classification where the Hough transform and the trimmed means methods are used to classify small sets of pixels. They compare the performance of these methods with the least squares error method, and they show that in the presence of outliers, the trimmed means method is far more reliable than the traditional least squares error method, and even when no outliers are present, its performance is comparable to that of the least squares error method. The method is exhaustively tested using simulated data, and it is also applied to real Landsat TM data for which ground data are available.

The authors introduce the concept of free angle to locate boundaries of textured regions in grey-level images. Using this with methods of mathematical morphology, the authors demonstrate that boundaries of all textured regions in remotely sensed images ran be located accurately.

A new method is proposed for processing randomly textured color images. The method is based on a bottom-up segmentation algorithm that takes into consideration both color and texture properties of the image. An LUV gradient is introduced, which provides both a color similarity measure and a basis for applying the watershed transform. The patches of watershed mosaic are merged according to their color contrast until a termination criterion is met. This criterion is based on the topology of the typical processed image. The resulting algorithm does not require any additional information, be it various thresholds, marker extraction rules, and suchlike, thus being suitable for automatic processing of color images. The algorithm is demonstrated within the framework of the problem of automatic granite inspection. The segmentation procedure has been found to be very robust, producing good results not only on granite images, but on the wide range of other noisy color images as well, subject to the termination criterion.

A post-processing scheme for edge detectors is described, that is based on the method of probabilistic relaxation where the binary relations between the possible edgels are explicitly modelled. The probability density functions of the errors in the measurements of these quantities have been rigorously derived from the assumed Gaussian distribution of errors in the image gray values.

We propose the use of the randomized Hough transform algorithm for the determination of the proportions of pure classes present in sets of mixed pixels, for large datasets (for which the deterministic Hough is prohibitively slow) and in the presence of outliers (i.e. in cases that the classical least square error method cannot cope). We demonstrate our results both with simulated and laboratory real data.

In this paper we investigate the relationship between fuzzy logic and neural networks. We show that fuzzy reasoning with general reasoning operators can be mapped on a multilayer perceptron architecture. This mapping allows us to give physical meaning to the weights of the neural network and may allow the user to bypass the training stage of the network if the necessary information is available. These ideas are exemplified with the help of a real problem from geography, concerned with the assessment of the regeneration potential of a burned forest for the optimal allocation of reforestation resources.

In previous work, a probabilistic method for matching of 2-D geometric features has been described. For this method, a noise model is required for the compatibility coefficients, which has hitherto been specified heuristically. In this paper, the authors show how an improved noise model can be derived, using as a basis the noise model of the feature measurements. The new noise model is shown to provide better performance than the heuristic one. Using a non-iterative version of the matching algorithm, a good solution was found using the new model, which was not the case for the old one. When an iterative version was used, both models generated good solutions, but the new model resulted in faster convergence. Furthermore, no training is needed to generate covariances for the new model. A means of explicitly modelling small scaling errors is also given. 1 Introduction Many authors have described methods for feature labelling based on probabilistic considerations, in which features extrac...

Probabilistic Relaxation with product support has been shown to have advantages over 'traditional ' probabilistic relaxation. However it is less well understood in the sense that a cost function is not known. In this paper we present a cost function. This greatly improves our understanding of probabilistic relaxation with product support, and also leads us to propose a new class of probabilistic relaxation algorithms. We investigate two applications. 1

In this paper, we develop the theory of probabilistic relaxation for matching features extracted from 2D images, derive as limiting cases the various heuristic formulae used by researchers in matching problems, and state the conditions under which they apply, We successfully apply our theory to the problem of matching and recognizing aerial road network images based on road network models and to the problem of edge matching in a stereo pair. For this purpose, each line network is represented by an attributed relational graph where each node is a straight line segment characterized by certain attributes and related with every other node via a set of binary relations.>

The 3D image reconstruction method based on Projective Geometry is very attractive for practical applications because it does not require any camera calibration. It requires only the knowledge of at least 8 reference points defining two planes. We investigate here the sensitivity of the results of this method to the measurement errors in the positions of the reference points both on the image plane and in the 3D world, and suggest some procedures which should be used in the practical applications of the method to avoid excessive error amplification.

In this paper, we present a novel algorithm which is based on the concept of free angle for the purpose of texture region boundary identification. The input to the algorithm is a grey level image. The output is a binary edge map marking the closed contour boundaries of the textured regions. The algorithm has been tested on several remotely sensed images and gives very good and robust results. 1

Using Markov random field (MRF) theory, a variety of computer vision problems can be modeled in terms of optimization based on the maximum a posteriori (MAP) criterion. The MAP configuration minimizes the energy of a posterior (Gibbs) distribution. When the label set is discrete, the minimization is combinatorial. This paper proposes to use the continuous relaxation labeling (RL) method for the minimization. The RL converts the original NP complete problem into one of polynomial complexity. Annealing may be combined into the RL process to improve the quality (globalness) of RL solutions. Performance comparison among four different RL algorithms is given.

This paper argues that the correct way to use a multiresolution approach to image restoration is to coarsen the posterior distribution of the image via the renormalization group transformation (RGT). It summarises the main results on RGT and points out the difficulties with it. It proposes an alternative transformation which does not have the full properties of the RGT but which preserves the global minimum of the cost function. The major contribution of the paper is to prove some theoretical properties of this transformation which show its significance and usefulness to image processing.

This paper shows the application of a very efficient multiresolution transformation which is related to the renormalization group approach of physics, to the problem of motion segmentation. The proposed approach is much faster and yields much better results than the full resolution approach.

Straight lines from a single perspective image are used to determine orientations of objects in an image scene using skew symmetry. The lines are detected using a Hough transform followed by an optimization algorithm in order to remove discretization errors in the line parameters. The use of these accurate line parameter estimates is demonstrated with some test images for which the plane orientations are known. It is shown that orientations from skew symmetry can be reliably determined.>

The authors present the theory of probabilistic relaxation for matching symbolic structures, derive as limiting cases the various heuristic formulas used by researchers in matching problems, and state the conditions under which they apply. They successfully apply the theory to the problem of matching and recognizing aerial road network images based on road network models and to the problem of edge matching in a stereo pair. For this purpose, each line network is represented by an attributed relational graph where each node is a straight line segment characterized by certain attributes and related with every other node via a set of binary relations.>

The authors show that a Hough transform algorithm can give sufficiently accurate estimates of line parameters to accurately locate vanishing points, even when they are several hundred pixels off the image. For this purpose, a 2-D robust kernel is used and the continuity of the accumulator array is exploited to optimize the parameter estimates. The algorithm is described in detail and results are shown on a variety of images, including synthetic images to determine quantitatively the accuracy of the vanishing point location by performing statistical tests.>

The application of multigrid methods to accelerating the minimisation of a nonconvex Gibbs potential requires that Gibbs parameters (GP's) be chosen at each level of coarsening in such a way that the long-range conditional probabilities of the associated 2D Markov random field are unchanged by coarsening. Formulae specifying how GP's transform from fine to coarse resolution exist for a restricted set of computationaly tractable cost functions and, while numerical routines solve the problem in principle, they are hopelessly slow for that class of spatially inhomogeneous GP's (site parameters) which depend on the image data. The authors provide a general theorem which specifies how site parameters coarsen for deterministic linear block coarsening. The theorem can be used to accelerate the annealing of realistic image models.>

It is advantageous to use edge orientation information from an edge detector when trying to find lines in an image using a Hough transform algorithm. Such methods have to balance between two competing problems: (1) to reduce interference between different line segments in the image; and (2) to allow for increased orientation uncertainties particularly near junctions of lines. To counter both these problems the authors show that a smooth voting kernel which is a function of differences both in orientation and distance from the line, can give superior results. They compare these results with the results obtained from using the standard Hough implementation as well as an implementation with a kernel which varies smoothly with separation but remains independent of differences in orientation.>

It is argued that the best way to model an edge is by assuming all ideal mathematical function passed through a low-pass filter and and immersed in noise. Using techniques similar to those developed by J. Canny (1983, 1986) and L.A. Spacek (1986), optimal filters are derived for ramp edges of various slopes. The optimal nonrecursive filter for ideal step edges is then derived as a limiting case of the filters for ramp edges. Because there are no step edges in images, edge detection is improved when the ramp filter is used instead of the filters developed for step edges. For practical purposes, some convolution masks are given which can be used directly for edge detection without the need to go into the details of the subject.>

Remotely-sensed data constitute a major potential source of input to geographical information systems (GIS)However, these data often have a relatively poor classification accuracy compared with that of the cartographic data from maps with which they may be combined in the course of GIS analysis. The possibility exists of using data sets (in the form of digital maps) resident within a GIS in order to improve this accuracy, before the classified image is incorporated into the GIS. Results are discussed from a British Alvey Information Technology project to develop a system for the knowledge-based segmentation and classification of remotely-sensed terrain images, in which the knowledge contained in digital map