Although supervised multi-view 3D reconstruction methods have achieved satisfying performance recently, there are major limitations such as high costs for 3D data collection and poor generalization to unseen scenes. Hence, unsupervised 3D reconstruction approaches based on photometric consistency are being explored. However, variations in lighting conditions among different views and reflective surfaces within a scene can undermine the reliability of these approaches. In this paper, we propose adaptive depth priors as pseudo-labels to guide the optimization process of self-supervised multiview stereo. First, sparse depth priors are generated based on the conventional structure from motion (SfM) and multi-view stereo (MVS) algorithms, which are then fed into a monocular depth estimation network to learn the adapted depth priors. Besides, a spatial-frequency fusion structure is designed to enhance global perception in the feature matching of MVS by combining local dependency from spatial domain with global contextual information in the frequency domain. Extensive experiments on DTU and Tanks & Temples datasets demonstrate that the proposed ADP-MVSNet achieves markedly improved results over the existing unsupervised approaches and even outperforms some supervised methods.

We propose a solution to Fourier ptychographic microscopy (FPM) by combining a no-reference image quality assessment module based on information entropy (IENR-IQA) and a physics-based neural network (PbNN) to achieve rapid reconstruction from multiple low-resolution images to highresolution images. This improves the reconstruction and makes it more generalizable and robust than the traditional FPM methods. Existing reconstruction methodologies are susceptible to systematic errors, including pupil aberration and light-emitting diode (LED) positional intensity discrepancies, which profoundly impact reconstruction clarity and color fidelity. In response, a PbNN featuring image quality evaluation is introduced for FPM reconstruction in this paper. A key innovation is incorporating the IENR-IQA module within a PbNN, to facilitate adaptive correction of spatial variations in LED intensity and position. In addition, the IENR-IQA module employs a fully connected layer to rectify pupil aberration. Rigorous simulations and experiments were conducted to validate the proposed method’s effectiveness and robustness. Experimental results demonstrate that the proposed IENR-IQA model can predict image quality well. When combined with PbNN in FPM, the image quality is improved compared to the existing physical neural networks, making the reconstruction results closer to human vision perception. The proposed IENR-IQA PbNN enhances the possibilities for applying the FPM technology in practice.

In histopathological image analysis, ability to learn representations from limited labeled data is crucial. A notable challenge is the scarcity of labeled pathological imagery, which has increasingly steered the focus towards self-supervised learning. However, existing methods often overlook several key factors: specific imaging mechanisms, high inter-class similarities, and considerable domain shifts. These oversights tend to compromise the quality of the learned class-level representations, particularly in the context of domain-level correlation information. To address these challenges, a novel self-supervised learning method, Stochastic Pathology Space Frequency Interpolation (SPACER), is introduced in this paper. The approach uniquely embeds pathology-specific imaging mechanisms and frequency-space information into the representation learning process. By doing so, SPACER can effectively navigate the subtle landscape of histopathological imagery. Its efficacy is demonstrated through extensive experiments. The proposed method outperforms current state-of-the-art self-supervised learning models on the UHU datasets by a significant margin: 5.06% on top-1 accuracy and 4.72% on dice coefficient in classification and segmentation tasks, respectively. The improvement shows the robustness of the proposed method in learning representations from histopathology imagery. It also shows improved scalability and generalizability on the Unitopath dataset. The deep mining of pathology-specific information by SPACER bridges the domain differences across different types of pathological images. This capability may also hold potentials for other types of medical images, which often exhibit inherent domain differences across clinics or institutes, hence presenting a broader applicability of the proposed method.

Abstract Deep network design is a fundamental challenge. A right trade‐off between depth and complexity of convolutional neural networks is of significant importance to applications in low‐level vision tasks. Wider feature maps could be beneficial to performance and generality but would increase computational complexity. In this paper, we rethink the balance between width of the feature maps and depth of the network especially for image restoration tasks including deblurring, dehazing, super‐resolution, and denoising. We explore a new approach to network structure by encouraging more depth to deal with restoration requirements while decreasing the width of some feature maps. Such a slimmer and deeper approach can enhance the performance while maintaining the same level of computational costs. We have experimentally evaluated the performances of the proposed approach on four image restoration tasks and obtained state‐of‐the‐art results on quantitative measures and qualitative assessments, demonstrating the effectiveness of the approach.

Abstract Most existing multi‐view stereo (MVS) methods fail to consider global context information in the stage of feature extraction and cost aggregation. As transformers have shown remarkable performance on various vision tasks due to their ability to perceive global contextual information, this paper proposes a transformer‐based feature enhancement network (TF‐MVSNet) to facilitate feature representation learning by combining local features (both 2D and 3D) with long‐range contextual information. To reduce memory consumption of feature matching, the cross‐attention mechanism is leveraged to efficiently construct 3D cost volumes under the epipolar constraint. Additionally, a colour‐guided network is designed to refine depth maps at a coarse stage, hence reducing incorrect depth predictions at a fine stage. Extensive experiments were performed on the DTU dataset and Tanks and Temples (T&T) benchmark and results are reported.

The utilisation of synthetic aperture radar (SAR) imagery for change detection can effectively circumvents the stringent limitations imposed by weather and lighting conditions, and is finding widespread applications in fields such as disaster monitoring and urban research. To address issues of edge blurring, severe noise interference and sample imbalance, an automated SAR change detection framework is proposed based on enhanced edge information and prototype constrained clustering. Firstly, a gradient-based neighbourhood ratio is designed to reinforce the edge information of the difference map, facilitating robust differential information representations. Subsequently, to obtain accurate samples in an unsupervised manner, we have developed prototype constrained hierarchical clustering for pre-classification. The quantity and quality of selected samples can be precisely guaranteed through the utilisation of histogram analysis and prototype constraints. In the sample learning and prediction phases, a class-balanced noise-tolerant change detection network is proposed that combines focal loss and mean absolute error loss, further tackling the sample imbalance issue, strengthening noise resistance and improving change detection accuracy. Comprehensive experimental results and analysis conducted on five benchmark datasets have validated the effectiveness and robustness of the proposed method.

This article is concerned with the problem of planning optimal maneuver trajectories and guiding the mobile robot toward target positions in uncertain environments for exploration purposes. A hierarchical deep learning-based control framework is proposed which consists of an upper level motion planning layer and a lower level waypoint tracking layer. In the motion planning phase, a recurrent deep neural network (RDNN)-based algorithm is adopted to predict the optimal maneuver profiles for the mobile robot. This approach is built upon a recently proposed idea of using deep neural networks (DNNs) to approximate the optimal motion trajectories, which has been validated that a fast approximation performance can be achieved. To further enhance the network prediction performance, a recurrent network model capable of fully exploiting the inherent relationship between preoptimized system state and control pairs is advocated. In the lower level, a deep reinforcement learning (DRL)-based collision-free control algorithm is established to achieve the waypoint tracking task in an uncertain environment (e.g., the existence of unexpected obstacles). Since this approach allows the control policy to directly learn from human demonstration data, the time required by the training process can be significantly reduced. Moreover, a noisy prioritized experience replay (PER) algorithm is proposed to improve the exploring rate of control policy. The effectiveness of applying the proposed deep learning-based control is validated by executing a number of simulation and experimental case studies. The simulation result shows that the proposed DRL method outperforms the vanilla PER algorithm in terms of training speed. Experimental videos are also uploaded, and the corresponding results confirm that the proposed strategy is able to fulfill the autonomous exploration mission with improved motion planning performance, enhanced collision avoidance ability, and less training time.

Abstract Over the last decade, convolutional neural networks (CNNs) have allowed remarkable advances in single image super-resolution (SISR). In general, recovering high-frequency features is crucial for high-performance models. High-frequency features suffer more serious damages than low-frequency features during downscaling, making it hard to recover edges and textures. In this paper, we attempt to guide the network to focus more on high-frequency features in restoration from both channel and spatial perspectives. Specifically, we propose a high-frequency channel attention (HFCA) module and a frequency contrastive learning (FCL) loss to aid the process. For the channel-wise perspective, the HFCA module rescales channels by predicting statistical similarity metrics of the feature maps and their high-frequency components. For the spatial perspective, the FCL loss introduces contrastive learning to train a spatial mask that adaptively assigns high-frequency areas with large scaling factors. We incorporate the proposed HFCA module and FCL loss into an EDSR baseline model to construct the proposed lightweight high-frequency channel contrastive network (HFCCN). Extensive experimental results show that it can yield markedly improved or competitive performances compared to the state-of-the-art networks of similar model parameters.

Person search is a practical but challenging task aiming to locate and identify the target person in unconstrained images. Most existing works are focused on people’s features while ignores scene features, which can help generate effective features for person identities. To address this issue, we present a Scene Contextual Enhanced Network(SCENet), by introducing attention refined scene contextual features to enhance person embedding. Also, a Scene Person Dissimilarity Loss is proposed to relieve the negative effects of irrelevant people in scene features. Experimental results on two person search benchmarks, i.e., CUHK-SYSU and PRW, demonstrate that our proposed model outperforms existing methods and achieves state-of-the-art performances.

In unsupervised domain adaptation (UDA), a target-domain model is trained by the supervised knowledge from a source domain. Although UDA has recently received much attention, most existing UDA methods have ignored the alignment in label space while mostly concentrating on alignment of feature space. Even worse, they have payed less attention to the dynamic relationship between domain alignment and discrimination, leading to degenerated performance. In this work, we propose a new kind of UDA through aligning in both the feature and label spaces (DAFL), in which a dynamic weight is designed and deployed between domain alignment and discrimination enhancement according to their conditions. Specifically, the cross-domain distribution divergence is reduced by the weighted class-level feature space alignment as well as the compacted and discriminative label space alignment. Furthermore, the balancing weight between adaptation alignment and discrimination enhancement is dynamically adjusted to regularize the adversarial domain adaptation. Then, the generalization ability of the DAFL model is enhanced by adding discrepant classification with theoretical analysis. Finally, extensive experiments validate effectiveness and superiority of the proposed approach.

Canonical correlation analysis (CCA) is a typical statistical model used to analyze the correlation components between different view representations of the same objects. When the label information is available with the data representations, CCA can be extended to its discriminative counterparts by incorporating supervision in the analysis. Although most discriminative variants of CCA have achieved improved results, nearly all of their objective functions are nonconvex, implying that optimal solutions are difficult to obtain. More important, that cross-view representations from the same sample should be consistent, that is, the cross-view semantic consistency has however not been modeled. To overcome these drawbacks, in this article, we propose a discriminant semantic correlation analysis (DSCA) model by modeling the cross-view semantic consistency for each object in the sample space rather than in the commonly used feature space. To boost the nonlinear discriminating capability of DSCA, we extend it from the Euclidean to the geodesic space by transforming the metric and incorporating both the cross-view semantic and representation correlation information and consequently obtain our final model with convex objective, namely, convex DSCA (C-DSCA). Finally, with extensive experiments and comparisons, we validate the effectiveness and superiority of the proposed method.

Given a photo of a subject, ability to generate a caricature image that captures distinct characteristics of the subject but with certain exaggeration of their prominent features is of fundamental importance to image processing and facial recognition. There are two main challenges in this task: shape exaggeration and style transfer. The former morphs and exaggerates key facial features of the subject, while the latter generates caricature images in a certain artistic style. In this paper, we propose a CAricature Style Transfer (CAST) framework for caricature generation. There are two modules in the proposed framework. The first is a geometric warping module. Different from the existing style transfer methods, we incorporate the Whitening and Coloring Transformation (WCT) in the geometric style transfer. The WCT is learned on photo and caricature landmarks or the caricature landmark space of a specific artist and is capable of transforming input photo landmarks to caricature landmarks. The second module is a texture style rendering module. We propose a new style transfer method by considering a semantic region-aligned style transfer via affinity constraint. Given a reference caricature image as the style reference, this module is capable of transferring styles between the same or similar semantic regions in caricatures and photos. Furthermore, it can transfer visual attributes of the reference caricatures (such as mouth shape and expressions) to the output caricatures. Experiments have shown desirable effects of the proposed method in transferring both the geometric and artistic texture styles of caricatures. Both qualitative and quantitative results show that the CAST framework is more effective compared than the state-of-the-art caricature generation methods.

Generative adversarial networks (GANs) have been widely adopted in many image processing tasks including restoration. In order to further improve quality of generated images, the training objective function needs to incorporate more constraints in addition to the adversarial loss. It can be straightforward to combine various losses in a linear fashion. However, hyperparameter fine-tuning and non-convex loss optimization are challenging problems when combining cost functions in such a manner. Here, we propose an efficient formulation of multiple loss components for training GANs. The proposed method, termed HypervolGAN, not only provides an efficient alternative for simultaneous cost optimization, but also boosts model performance in terms of improving generated image quality without excess computation. We further introduce two image-quality-measure based loss components to the GANs specifically for image restoration. Extensive evaluations and results on various benchmark datasets validate the effectiveness of the proposed methods.

The spatial attention mechanism captures long-range dependencies by aggregating global contextual information to each query location, which is beneficial for semantic segmentation. In this paper, we present a sparse spatial attention network (SSANet) to improve the efficiency of the spatial attention mechanism without sacrificing the performance. Specifically, a sparse non-local (SNL) block is proposed to sample a subset of key and value elements for each query element to capture long-range relations adaptively and generate a sparse affinity matrix to aggregate contextual information efficiently. Experimental results show that the proposed approach outperforms other context aggregation methods and achieves state-of-the-art performance on the Cityscapes, PASCAL Context and ADE20K datasets.

Graph convolutional networks (GCNs) have achieved great success on dealing with data of non-Euclidean structures. Their success directly attribute to effective fitting graph structures to data such as in social media and knowledge databases. For image processing applications, the use of graph structures and GCNs have not been fully explored. In this paper, we propose a novel encoder-decoder network with added graph convolutions by converting feature maps to vertexes of a pre-generated graph to synthetically construct graph-structured data. By doing this, we inexplicitly apply graph Laplacian regularization to the feature maps, making them more structured. The experiments show it significantly boosts performance for the task of image restoration, including deblurring and super-resolution. We believe that it opens up opportunities for GCN-based approaches in many applications.

Although the least-squares regression (LSR) has achieved great success in regression tasks, its discriminating ability is limited since the margins between classes are not specially preserved. To mitigate this issue, dragging techniques have been introduced to remodel the regression targets of LSR. Such variants have gained certain performance improvement, but their generalization ability is still unsatisfactory when handling real data. This is because structure-related information, which is typically contained in the data, is not exploited. To overcome this shortcoming, in this article, we construct a multioutput regression model by exploiting the intraclass correlations and input-output relationships via a structure matrix. We also discriminatively enlarge the regression margins by embedding a metric that is guided automatically by the training data. To better handle such structured data with ordinal labels, we encode the model output as cumulative attributes and, hence, obtain our proposed model, termed structure-exploiting discriminative ordinal multioutput regression (SEDOMOR). In addition, to further enhance its distinguishing ability, we extend the SEDOMOR to its nonlinear counterparts with kernel functions and deep architectures. We also derive the corresponding optimization algorithms for solving these models and prove their convergence. Finally, extensive experiments have testified the effectiveness and superiority of the proposed methods.

Explainability is an important factor to drive user trust in the use of neural networks for tasks with material impact. However, most of the work done in this area focuses on image analysis and does not take into account 3D data. We extend the saliency methods that have been shown to work on image data to deal with 3D data. We analyse the features in point clouds and voxel spaces and show that edges and corners in 3D data are deemed as important features while planar surfaces are deemed less important. The approach is model-agnostic and can provide useful information about learnt features. Driven by the insight that 3D data is inherently sparse, we visualise the features learnt by a voxel-based classification network and show that these features are also sparse and can be pruned relatively easily, leading to more efficient neural networks. Our results show that the Voxception-ResNet model can be pruned down to 5% of its parameters with negligible loss in accuracy.

Massively complex systems, such as social networks (Camacho, Panizo-LLedot, Bello-Orgaz, Gonzalez-Pardo, & Cambria, 2020; Lara-Cabrera et al., 2017), renewable energy problems (Twidell & Weir, 2015), or Internet-of-Things problems (Lin et al., 2017), generate massive amounts of data. These massively complex systems have attracted the attention of both industrial and research communities, because the analysis of data can generate valuable knowledge about the specific domain. But at the same time, the amount of data generated and the complexity of the problems mean that classical algorithms and approaches do not provide suitable solutions. In this case, it is quite common for computational intelligence (CI) techniques to extract the knowledge. CI can be defined as a set of bio-inspired research areas focused on the study of adaptive mechanisms to enable, or facilitate, intelligent behaviour in complex and changing environments. There are several research fields that compose CI, including swarm intelligence (Gonzalez-Pardo, Jung, & Camacho, 2017), and evolutionary computation (Salcedo-Sanz, Ortiz-Garcýa, Ángel M. Pérez-Bellido, Portilla-Figueras, & Prieto, 2011). This special issue is focused on the application of bio-inspired algorithms to massively complex systems, ranging from concepts and theoretical developments to advances technologies and innovative applications. This special issue welcomed submissions of original papers introducing research results on all the aspects covering the application of CI algorithms to massively complex systems, ranging from concepts and theoretical developments to advanced technologies and innovative applications. This issue presents expanded versions of the best papers presented at the 19th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL 2018), which was held in Madrid (Spain). As the special issue editors, we would like to take this opportunity to thank the various authors for their papers and the reviewers for their work. We are also grateful to Jon Hall, Editor-in-Chief of the Wiley journal Expert Systems. We would like to particularly thank the IDEAL'18 programme committee members for their hard work and dedication.

LiDAR provides highly accurate 3D point clouds. However, data needs to be manually labelled in order to provide subsequent useful information. Manual annotation of such data is time consuming, tedious and error prone, and hence in this paper we present three automatic methods for annotating trees in LiDAR data. The first method requires high density point clouds and uses certain LiDAR data attributes for the purpose of tree identification, achieving almost 90% accuracy. The second method uses a voxel-based 3D Convolutional Neural Network on low density LiDAR datasets and is able to identify most large trees accurately but struggles with smaller ones due to the voxelisation process. The third method is a scaled version of the PointNet++ method and works directly on outdoor point clouds and achieves an F_score of 82.1% on the ISPRS benchmark dataset, comparable to the state-of-the-art methods but with increased efficiency.

Canonical correlation analysis (CCA) is a typical and useful learning paradigm in big data analysis for capturing correlation across multiple views of the same objects. When dealing with data with additional ordinal information, traditional CCA suffers from poor performance due to ignoring the ordinal relationships within the data. Such data is becoming increasingly common, as either temporal or sequential information is often associated with the data collection process. To incorporate the ordinal information into the objective function of CCA, the so-called ordinal discriminative CCA has been presented in the literature. Although ordinal discriminative CCA can yield better ordinal regression results, its performance deteriorates when data is corrupted with noise and outliers, as it tends to smear the order information contained in class centers. To address this issue, in this article we construct a robust manifold-preserved ordinal discriminative correlation regression (rmODCR). The robustness is achieved by replacing the traditional ( l 2 -norm) class centers with l p -norm centers, where p is efficiently estimated according to the moments of the data distributions, as well as by incorporating the manifold distribution information of the data in the objective optimization. In addition, we further extend the robust manifold-preserved ordinal discriminative correlation regression to deep convolutional architectures. Extensive experimental evaluations have demonstrated the superiority of the proposed methods.

In this paper, we address the semantic segmentation task with a deep network that combines contextual features and spatial information. The proposed Cross Attention Network is composed of two branches and a Feature Cross Attention (FCA) module. Specifically, a shallow branch is used to preserve low-level spatial information and a deep branch is employed to extract high-level contextual features. Then the FCA module is introduced to combine these two branches. Different from most existing attention mechanisms, the FCA module obtains spatial attention map and channel attention map from two branches separately, and then fuses them. The contextual features are used to provide global contextual guidance in fused feature maps, and spatial features are used to refine localizations. The proposed network outperforms other real-time methods with improved speed on the Cityscapes and CamVid datasets with lightweight backbones, and achieves state-of-the-art performance with a deep backbone.

Facial expression synthesis has drawn increasing attention in computer vision, graphics and animation. Recently, generative adversarial nets (GANs) have become a new perspective for face synthesis and have had remarkable success in generating photorealistic images and image‐to‐image translation. In this study, the authors present an appearance‐based facial expression synthesis framework, ApprGAN, by combining shape and texture and introducing cycle consistency and identity mapping into the adversarial learning. Specifically, given an input face image, a pair of shape and texture generators are trained for synthetic shape deformation and expression detail generation, respectively. Extensive experiments on expression synthesis and cross‐database synthesis were conducted, together with comparisons with the existing methods. Results of expression synthesis and quantitative verification on various databases show the effectiveness of ApprGAN in synthesising photorealistic and identity‐preserving expressions and its marked improvement over the existing methods.

We present here an unsupervised approach to learning suitable features for a deep learning framework applied to image classification. PCANet was introduced as a simple and efficient baseline for deep learning approaches which used cascaded principle component analysis (PCA) derived filter banks, as well as other simple image processing elements such as binary hashing and blockwise histograms. This was followed by DCTNet which used discrete cosine transform (DCT) filter banks as a learning-free alternative. In this paper we propose SOMNet which uses self-organizing map (SOM) based filters offering a non-orthogonal alternative to PCANet providing comparable performance. It is well established that SOM is a non-linear version of PCA but does not suffer from the same constraints. We also show that through the use of a simple trick in the binarization process results in a dramatic reduction in the dimension of the final feature vector, thus allowing the utilization of more filters which could lead to deeper and more complex structures in further work. We also demonstrate the results of a hybrid methodology that clusters generative Markov random fields (MRF) as filters which provides more diverse features in a data driven approach to deep learning.

Standard neural network architectures are non-linear only by virtue of a simple element-wise activation function, making them both brittle and excessively large. In this paper, we consider methods for making the feed-forward layer more flexible while preserving its basic structure. We develop simple drop-in replacements that learn to adapt their parameterization conditional on the input, thereby increasing statistical efficiency significantly. We present an adaptive LSTM that advances the state of the art for the Penn Treebank and Wikitext-2 word-modeling tasks while using fewer parameters and converging in half as many iterations.

Many real-world problems require modeling and forecasting of time series, such as weather temperature, electricity demand, stock prices and foreign exchange (FX) rates. Often, the tasks involve predicting over a long-term period, e.g. several weeks or months. Most existing time series models are inheritably for one-step prediction, that is, predicting one time point ahead. Multi-step or long-term prediction is difficult and challenging due to the lack of information and uncertainty or error accumulation. The main existing approaches, iterative and independent, either use one-step model recursively or treat the multi-step task as an independent model. They generally perform poorly in practical applications. In this paper, as an extension of the self-organizing mixture autoregressive (AR) model, the varied length mixture (VLM) models are proposed to model and forecast time series over multi-steps. The key idea is to preserve the dependencies between the time points within the prediction horizon. Training data are segmented to various lengths corresponding to various forecasting horizons, and the VLM models are trained in a self-organizing fashion on these segments to capture these dependencies in its component AR models of various predicting horizons. The VLM models form a probabilistic mixture of these varied length models. A combination of short and long VLM models and an ensemble of them are proposed to further enhance the prediction performance. The effectiveness of the proposed methods and their marked improvements over the existing methods are demonstrated through a number of experiments on synthetic data, real-world FX rates and weather temperatures.

Heterogeneous face recognition deals with matching face images from different modalities or sources. The main challenge lies in cross-modal differences and variations and the goal is to make cross-modality separation among subjects. A margin-based cross-modality metric learning (MCM2L) method is proposed to address the problem. A cross-modality metric is defined in a common subspace where samples of two different modalities are mapped and measured. The objective is to learn such metrics that satisfy the following two constraints. The first minimizes pairwise, intrapersonal cross-modality distances. The second forces a margin between subject specific intrapersonal and interpersonal cross-modality distances. This is achieved by defining a hinge loss on triplet-based distance constraints for efficient optimization. It allows the proposed method to focus more on optimizing distances of those subjects whose intrapersonal and interpersonal distances are hard to separate. The proposed method is further extended to a kernelized MCM2L (KMCM2L). Both methods have been evaluated on an ID card face dataset and two other cross-modality benchmark datasets. Various feature extraction methods have also been incorporated in the study, including recent deep learned features. In extensive experiments and comparisons with the state-of-the-art methods, the MCM2L and KMCM2L methods achieved marked improvements in most cases.

Cross-modal metric learning (CML) deals with learning distance functions for cross-modal data matching. The existing methods mostly focus on minimizing a loss defined on sample pairs. However, the numbers of intraclass and interclass sample pairs can be highly imbalanced in many applications, and this can lead to deteriorating or unsatisfactory performances. The area under the receiver operating characteristic curve (AUC) is a more meaningful performance measure for the imbalanced distribution problem. To tackle the problem as well as to make samples from different modalities directly comparable, a CML method is presented by directly maximizing AUC. The method can be further extended to focus on optimizing partial AUC (pAUC), which is the AUC between two specific false positive rates (FPRs). This is particularly useful in certain applications where only the performances assessed within predefined false positive ranges are critical. The proposed method is formulated as a log-determinant regularized semidefinite optimization problem. For efficient optimization, a minibatch proximal point algorithm is developed. The algorithm is experimentally verified stable with the size of sampled pairs that form a minibatch at each iteration. Several data sets have been used in evaluation, including three cross-modal data sets on face recognition under various scenarios and a single modal data set, the Labeled Faces in the Wild. Results demonstrate the effectiveness of the proposed methods and marked improvements over the existing methods. Specifically, pAUC-optimized CML proves to be more competitive for performance measures such as Rank-1 and verification rate at FPR = 0.1%.

Heterogeneous face recognition aims to identify or verify person identity by matching facial images of different modalities. In practice, it is known that its performance is highly influenced by modality inconsistency, appearance occlusions, illumination variations and expressions. In this paper, a new method named as ensemble of sparse cross-modal metrics is proposed for tackling these challenging issues. In particular, a weak sparse cross-modal metric learning method is firstly developed to measure distances between samples of two modalities. It learns to adjust rank-one cross-modal metrics to satisfy two sets of triplet based cross-modal distance constraints in a compact form. Meanwhile, a group based feature selection is performed to enforce that features in the same position of two modalities are selected simultaneously. By neglecting features that attribute to "noise" in the face regions (eye glasses, expressions and so on), the performance of learned weak metrics can be markedly improved. Finally, an ensemble framework is incorporated to combine the results of differently learned sparse metrics into a strong one. Extensive experiments on various face datasets demonstrate the benefit of such feature selection especially when heavy occlusions exist. The proposed ensemble metric learning has been shown superiority over several state-of-the-art methods in heterogeneous face recognition.

Neural networks have been widely applied to time series prediction over past few decades. Generally, applications of them restrict to causal models where current values are dependent on past values. In contrast, a non-causal neural network is proposed in this paper to deal with time series prediction by allowing dependence on future values. Both past and future values are used together for training and prediction. In prediction, future values are the expected values of training samples. In addition, weightings of the past and future values are incorporated into the network to improve prediction performance. Experimental results on benchmark and FX time series show that the proposed network is effective.

In this paper, a novel method termed Multi-Instance Dictionary Learning (MIDL) is presented for detecting abnormal events in crowded video scenes. With respect to multi-instance learning, each event (video clip) in videos is modeled as a bag containing several sub-events (local observations); while each sub-event is regarded as an instance. The MIDL jointly learns a dictionary for sparse representations of sub-events (instances) and multi-instance classifiers for classifying events into normal or abnormal. We further adopt three different multi-instance models, yielding the Max-Pooling-based MIDL (MP-MIDL), Instance-based MIDL (Inst-MIDL) and Bag-based MIDL (Bag-MIDL), for detecting both global and local abnormalities. The MP-MIDL classifies observed events by using bag features extracted via max-pooling over sparse representations. The Inst-MIDL and Bag-MIDL classify observed events by the predicted values of corresponding instances. The proposed MIDL is evaluated and compared with the state-of-the-art methods for abnormal event detection on the UMN (for global abnormalities) and the UCSD (for local abnormalities) datasets and results show that the proposed MP-MIDL and Bag-MIDL achieve either comparable or improved detection performances. The proposed MIDL method is also compared with other multi-instance learning methods on the task and superior results are obtained by the MP-MIDL scheme.

In this paper, a structurally enhanced incremental neural learning technique is proposed to learn a discriminative codebook representation of images for effective image classification applications. In order to accommodate the relationships such as structures and distributions among visual words into the codebook learning process, we develop an online codebook graph learning method based on a novel structurally enhanced incremental learning technique, called as "visualization-induced self-organized incremental neural network (ViSOINN)". The hidden structural information in the images is embedded into the graph representation evolving dynamically with the adaptive and competitive learning mechanism. Afterwards, image features can be coded using a sub-graph extraction process based on the learned codebook graph, and a classifier is subsequently used to complete the image classification task. Compared with other codebook learning algorithms originated from the classical Bag-of-Features (BoF) model, ViSOINN holds the following advantages: (1) it learns codebook efficiently and effectively from a small training set; (2) it models the relationships among visual words in metric scaling fashion, so preserving high discriminative power; (3) it automatically learns the codebook without a fixed pre-defined size; and (4) it enhances and preserves better the structure of the data. These characteristics help to improve image classification performance and make it more suitable for handling large-scale image classification tasks. Experimental results on the widely used Caltech-101 and Caltech-256 benchmark datasets demonstrate that ViSOINN achieves markedly improved performance and reduces the computational cost considerably.

Novelty detection, or one-class classification, is of particular use in the analysis of high-integrity systems, in which examples of failure are rare in comparison with the number of examples of stable behaviour, such that a conventional multi-class classification approach cannot be taken. Support Vector Machines (SVMs) are a popular means of performing novelty detection, and it is conventional practice to use a train-validate-test approach, often involving cross-validation, to train the one-class SVM, and then select appropriate values for its parameters. An alternative method, used with multi-class SVMs, is to calibrate the SVM output into conditional class probabilities. A probabilistic approach offers many advantages over the conventional method, including the facility to select automatically a probabilistic novelty threshold. The contributions of this paper are (i) the development of a probabilistic calibration technique for one-class SVMs, such that on-line novelty detection may be performed in a probabilistic manner; and (ii) the demonstration of the advantages of the proposed method (in comparison to the conventional one-class SVM methodology) using case studies, in which one-class probabilistic SVMs are used to perform condition monitoring of a high-integrity industrial combustion plant, and in detecting deterioration in patient physiological condition during patient vital-sign monitoring.

This paper presents a novel model for performing classification and visualization of high-dimensional data by means of combining two enhancing techniques. The first is a semi-supervised learning, an extension of the supervised learning used to incorporate unlabeled information to the learning process. The second is an ensemble learning to replicate the analysis performed, followed by a fusion mechanism that yields as a combined result of previously performed analysis in order to improve the result of a single model. The proposed learning schema, termed S(2)-Ensemble, is applied to several unsupervised learning algorithms within the family of topology maps, such as the Self-Organizing Maps and the Neural Gas. This study also includes a thorough research of the characteristics of these novel schemes, by means quality measures, which allow a complete analysis of the resultant classifiers from the viewpoint of various perspectives over the different ways that these classifiers are used. The study conducts empirical evaluations and comparisons on various real-world datasets from the UCI repository, which exhibit different characteristics, so to enable an extensive selection of situations where the presented new algorithms can be applied.

The empirical mode decomposition (EMD) method can adaptively decompose a non-stationary time series into a number of amplitude or frequency modulated functions known as intrinsic mode functions. This paper combines the EMD method with information analysis and presents a framework of information-preserving EMD. The enhanced EMD method has been exploited in the analysis of neural recordings. It decomposes a signal and extracts only the most informative oscillations contained in the non-stationary signal. Information analysis has shown that the extracted components retain the information content of the signal. More importantly, a limited number of components reveal the main oscillations presented in the signal and their instantaneous frequencies, which are not often obvious from the original signal. This information-coupled EMD method has been tested on several field potential datasets for the analysis of stimulus coding in visual cortex, from single and multiple channels, and for finding information connectivity among channels. The results demonstrate the usefulness of the method in extracting relevant responses from the recorded signals. An investigation is also conducted on utilizing the Hilbert phase for cases where phase information can further improve information analysis and stimulus discrimination. The components of the proposed method have been integrated into a toolbox and the initial implementation is also described.

Local binary pattern (LBP) has recently been proposed for texture analysis and local feature description and has also been applied to face recognition with promising results. However, besides the descriptors, a suitable similarity measure that can efficiently learn to distinguish facial features is also important. In this paper, a novel framework for robust face recognition is presented that considers both local and global features by using multi-resolution LBP descriptors. The framework can tolerate variations in expression, lighting condition and occlusion. A weighted distance measure is used to learn the dissimilarity between sets of LBP features. We formulate the distance function as a conditionally positive semi-definite (CPD) kernel, thus making it suitable for kernel-based algorithms such as support vector machines (SVMs) whose optimal solutions are guaranteed. We show that by defining it in a Hilbert space, the proposed CPD kernel has advantages over traditional methods computing the l2distances in the Euclidean space. The experiments show that the approach is efficient and significantly outperforms the current state-of-the-art methods on the publicly available AR face database.

Data clustering has been a major research and application topic in data mining. The self-organizing map (SOM) has been widely applied to tasks including multivariate data visualization and clustering. SOM not only quantizes the input data but also enables visual display of data, a property that does not exist in most clustering algorithms. In the past decade many developments have reported towards to mining useful information from a trained map. Most of them use post-processing methods in a two- or three-step procedure to enable finding clusters as contiguous regions on the map. The basic assumption relies on the data density approximation by the neurons through the unsupervised learning. By analyzing neighboring neurons and their relations and activities it is possible to draw, in many cases, the geometry of clusters. This paper discusses issues related to SOM clustering and segmentation with morphological image processing methods, such as filtering and watershed transform. It also briefly reviews SOM clustering related literature, such as surface-based and clustering (hierarchical and partitioning) algorithms. A new gradient-based visualization matrix is presented and results of benchmark data sets are described.

In order to meet the constantly increasing demand for ubiquitous, mobile access to the internet, next-generation mobile broadband communications systems based on OFDMA, such as IEEE 802.16 m, require a significant performance increase over previous generation systems, such as IEEE 802.16e-2005, particularly in cell-edge and average spectral efficiency. In this paper, we address the downlink adaptive frequency reuse (AFR) and multi-user MIMO (MU-MIMO) techniques which are considered to be the most promising candidates for meeting the requirements on cell-edge and average spectral efficiency of next-generation mobile broadband systems.

Principal component analysis (PCA) has long been a simple, efficient technique for dimensionality reduction. However, many nonlinear methods such as local linear embedding and curvilinear component analysis have been proposed for increasingly complex nonlinear data recently. In this paper, we investigate and compare linear PCA and various nonlinear methods for face recognition. Results drawn from experiments on real-world face databases show that both linear and nonlinear methods yield similar performance and differences in classification rate are insignificant to conclude which method is always superior. A nonlinearity measure is derived to quantify the degree of nonlinearity of a data set in the reduced subspace. It can be used to indicate the effectiveness of nonlinear or linear dimensionality reduction.

This paper describes a topological clustering of synchronous spike trains recorded in rat somatosensory cortex in response to sinusoidal vibrissal stimulations characterized by different frequencies and amplitudes. Discrete spike trains are first interpreted as continuous synchronous activities by a smoothing filter such as causal exponential function. Then clustering is performed using the self-organizing map, which yields topologically ordered clusters of responses with respect to the stimuli. The grouping is formed mainly along the product of amplitude and frequency of the stimuli. This result coincides with the result obtained previously using mutual information analysis on the same data set. That is, the response is proportional in logarithm to the energy of the vibration. It suggests that such clustering can naturally find underlying stimulus-response patterns and it also seems to associate the spike-count based mutual information decoding with temporal patterns of the neuronal activities. The study also shows that causal decaying exponential kernel is better than noncausal Gaussian kernel in interpreting the discrete spike trains into continues ones and produces better clusters.

Modelling non-stationary time series has been a difficult task for both parametric and nonparametric methods. One promising solution is to combine the flexibility of nonparametric models with the simplicity of parametric models. In this paper, the self-organising mixture autoregressive (SOMAR) network is adopted as a such mixture model. It breaks time series into underlying segments and at the same time fits local linear regressive models to the clusters of segments. In such a way, a global non-stationary time series is represented by a dynamic set of local linear regressive models. Neural gas is used for a more flexible structure of the mixture model. Furthermore, a new similarity measure has been introduced in the self-organising network to better quantify the similarity of time series segments. The network can be used naturally in modelling and forecasting non-stationary time series. Experiments on artificial, benchmark time series (e.g. Mackey-Glass) and real-world data (e.g. numbers of sunspots and Forex rates) are presented and the results show that the proposed SOMAR network is effective and superior to other similar approaches.

The self-organizing map (SOM) and some of its variants such as visualization induced SOM (ViSOM) have been seen to yield similar results to multidimensional scaling (MDS). However the exact connection has yet been established, though similar topographic mapping results have been shown in several studies. In this paper we provide a review on the SOM and its cost function and topological measures. Then we examine its relationship with MDS from their cost functions in the aspect of data visualization. The SOM is shown to produce a quantized, qualitative scaling and while the ViSOM a quantitative or metric scaling. The SOM can also be regarded as a generalized MDS to relate two metric spaces by forming a topological mapping between them. The connection between MDS and principal manifolds is also discussed.

In this paper a novel approach is introduced for modeling and clustering gene expression time-series. The radial basis function neural networks have been used to produce a generalized and smooth characterization of the expression time-series. A co-expression coefficient is defined to evaluate the similarities of the models based on their temporal shapes and the distribution of the time points. The profiles are grouped using a fuzzy clustering algorithm incorporated with the proposed co-expression coefficient metric. The results on artificial and real data are presented to illustrate the advantages of the metric and method in grouping temporal profiles. The proposed metric has also been compared with the commonly used correlation coefficient under the same procedures and the results show that the proposed method produces better biologically relevant clusters.

We present a new method for content management and knowledge discovery using a topology-preserving neural network. The method, termed topological organization of content (TOC), can generate a taxonomy of topics from a set of unannotated, unstructured documents. The TOC consists of a hierarchy of self-organizing growing chains (GCs), each of which can develop independently in terms of size and topics. The dynamic development process is validated continuously using a proposed entropy-based Bayesian information criterion (BIC). Each chain meeting the criterion spans child chains, with reduced vocabularies and increased specializations. This results in a topological tree hierarchy, which can be browsed like a table of contents directory or web portal. A brief review is given on existing methods for document clustering and organization, and clustering validation measures. The proposed approach has been tested and compared with several existing methods on real world web page datasets. The results have clearly demonstrated the advantages and efficiency in content organization of the proposed method in terms of computational cost and representation. The TOC can be easily adapted for large-scale applications. The topology provides a unique, additional feature for retrieving related topics and confining the search space.

Gene expression time-series are discrete, noisy, short and usually unevenly sampled. Most of the existing methods used to compare expression profiles, operate directly on the time points. While modelling, the profiles can lead to more generalised, smooth characterisation of gene expressions. In this paper, a radial basis function neural network is employed to model gene expression time-series. The orthogonal least square method, used for selection of centres, is further combined with a width optimisation scheme. The experiments on a number of expression datasets have shown the advantages of the approach in terms of generalisation and approximation. The results on known datasets have indeed coincided with biological interpretations.

Fueled by the explosive growth of the Internet, applications are demanding higher data rates and better services. Given the scarcity of radio resources, higher network capacities need to be achieved through more efficient use of the available bandwidth. Current cellular networks utilize frequency planning schemes that are optimized for circuit-switched applications, and thus is inherently problematic for future wireless packet networks with bursty, high peak-rate traffics. Random access schemes such as the ALOHA are seen as better solutions for packet networks. However, co-channel interference may significantly reduce the network throughput when the multicell load is heavy. In this paper, we propose a distributed rate adaptive packet access (DRAPA) scheme to combine the advantages of rate adaptation (in circuit-switched networks) and random access (in packet-switched networks). In particular, DRAPA allows terminal stations to transmit packets in random access fashion in the presence of brusty interference from neighboring cells. The packet code rate is adjusted according to interference level so that the retransmisson is controlled at an acceptable level. The DRAPA scheme subsumes two traditional schemes as the extreme cases, and has superior performance over the traditional schemes in terms of throughput and stability.

When used for visualization of high-dimensional data, the self-organizing map (SOM) requires a coloring scheme, such as the U-matrix, to mark the distances between neurons. Even so, the structures of the data clusters may not be apparent and their shapes are often distorted. In this paper, a visualization-induced SOM (ViSOM) is proposed to overcome these shortcomings. The algorithm constrains and regularizes the inter-neuron distance with a parameter that controls the resolution of the map. The mapping preserves the inter-point distances of the input data on the map as well as the topology. It produces a graded mesh in the data space such that the distances between mapped data points on the map resemble those in the original space, like in the Sammon mapping. However, unlike the Sammon mapping, the ViSOM can accommodate both training data and new arrivals and is much simpler in computational complexity. Several experimental results and comparisons with other methods are presented.

Efficient exploitation of spatial diversity is fundamentally important to resource critical wireless applications (Tsoulos 1999). In this paper, we first study the performance of intelligent scheduling for space-division multiple-access (SDMA) wireless networks (Suard 1998, Farsakh 1998). Based on the existing scheme, we propose a new medium access protocol (MAC) for multimedia SDMA/time-division multiple-access (TDMA) packet networks (Xu 1994, Ward 1993). The improved protocol performs scheduling based on users' spatial characteristics and quality-of-service parameters to achieve throughput multiplication and packet delay reduction. Performance of SDMA with scheduling is evaluated under mixed audio and data traffic patterns and results show that significant improvement in network performance can be achieved under the new protocol.

Multicarrier direct-sequence code-division multiple access (MC-DS-CDMA) has emerged recently as a promising candidate for the next generation broad-band mobile networks. We consider the design of multiuser receivers for MC-DS-CDMA communications over fading channels. We present a class of spreading codes that enables the simple despreading-combining receiver to achieve the performance of the optimum multiuser linear receiver. These codes are shown to be optimum for independent fading channels under a code design criterion derived. Also derived are analytic solutions of optimum spreading codes for any given channel fading statistics. Simulation results are provided to demonstrate the significant gains in performance and simplicity due to the proposed techniques.

A self-organizing mixture network (SOMN) is derived for learning arbitrary density functions. The network minimizes the Kullback-Leibler information metric by means of stochastic approximation methods. The density functions are modeled as mixtures of parametric distributions. A mixture needs not to be homogenous, i.e., it can have different density profiles. The first layer of the network is similar to Kohonen's self-organizing map (SOM), but with the parameters of the component densities as the learning weights. The winning mechanism is based on maximum posterior probability, and updating of the weights is limited to a small neighborhood around the winner. The second layer accumulates the responses of these local nodes, weighted by the learned mixing parameters. The network possesses a simple structure and computational form, yet yields fast and robust convergence. The network has a generalization ability due to the relative entropy criterion used. Applications to density profile estimation and pattern classification are presented. The SOMN can also provide an insight to the role of neighborhood function used in the SOM.

In this paper we present a novel loading algorithm for OFDM-based multiuser communication system to maximize the total system throughput while satisfying the total power and users' rate constraints. The new scheme determines the subcarrier, bit, and power allocation by decoupling an NP-hard combinatorial problem into two steps: (1) resource allocation (how much power and how many subcarriers for each user) based on users' average channel gains and their rate requirements; and (2) subcarrier assignment and bit loading based on users' channel profiles across all subcarriers. Compared to existing iterative methods, the two-step approach offers comparable capacity gain with much lower complexity.

Efficient exploitation of spatial diversity is fundamentally important to resource critical wireless applications. A medium access protocol (MAC) for space division multiple access (SDMA) packet networks is proposed. The new protocol performs scheduling based on users' spatial characteristics and quality-of-service (QoS) requirements to achieve throughput multiplication and reduction of packet delays. The system's performance is evaluated under mixed audio and data traffic patterns and the results show that the new protocol can lead to a significant improvement in network performance.

Hyvarinen et al. (2000) have developed an ICA based method for image denoising. The major advantage of their method is that the transformation matrix can by adjusted to suit the available data. However, in their method, the transformation matrix and shrinkage parameters need to be learned from noise-free data. In this paper, we propose a simplified shrinkage scheme, which has only one heuristic control parameter. Experimental results show that the ICA based method with this new shrinkage scheme achieves comparable performance to that of Hyvarinen et al.

We consider the design of multiuser receivers for improving the performance of MC DS CDMA communications over fading channels. We present a class of spreading codes that reduces the dimension of MMSE receiver and ML receiver without any performance loss. These codes are shown to be optimum for independent fading channels under a code design criterion derived. Simulation results are provided to demonstrate the significant gains in performance and simplicity due to the proposed techniques.

The self-organising mixture network (SOMN) is a learning algorithm for mixture densities, derived from minimising the Kullback-Leibler information by means of stochastic approximation methods. It has been shown the SOMN converges faster than the EM-based algorithms and generalises better as it is based on the expected likelihood rather than the sample likelihood. The derived algorithm has similar updating forms to the self-organising map (SOM), thus reveals the mixture interpreter role of the neighbourhood function used in the SOM. When the sample set is small, overfitting problems often occur in most algorithms. Further improvement can be achieved by averaging ensembles of the SOMNs. The algorithms have been applied to both experimental data and real-world problems. The results show that smoothed mixtures with improved accuracy have been obtained. Estimation variance has been reduced.

In this paper an analysis of the statistical and the convergence properties of Kohonen's self-organizing map of any dimension is presented. Every feature in the map is considered as a sum of a number of random variables. We extend the Central Limit Theorem to a particular case, which is then applied to prove that the feature space during learning tends to multiple gaussian distributed stochastic processes, which will eventually converge in the mean-square sense to the probabilistic centers of input subsets to form a quantization mapping with a minimum mean squared distortion either globally or locally. The diminishing effect, as training progresses, of the initial states on the value of the feature map is also shown.

A combination of the self-organising principle and a relaxation method is proposed for the unsupervised segmentation of textured images. There are two phases to this segregation. The first one is region-based and provides a coarse rapid segmentation. A hierarchical self-organising learning structure together with a Markov random field (MRF) model-based LS estimator is used for parameter estimation and classification of different regions from a randomly located window, which shrinks with time. A globally correct separation of different regions has been always formed during extensive experiments. These results are used as the initial states for the second phase processing, which is boundary-based. A relaxation method, similar to the Metropolis algorithm, is used for boundary improvement. Over a small area of original boundary, a new boundary is randomly generated, which accepted or rejected according to the local energy change. Experiment results on both synthetic and real textured images are presented.>