Pneumonia is an acute respiratory infection, posing a serious threat to health and lives. Lung ultrasound (LUS), as a non-invasive and rapid imaging technique, can monitor real-time changes in lung, providing valuable assistance in clinical diagnosis. However, most LUS studies are limited to frame-level analysis and ignore respiratory cycle changes, leading to diagnostic errors. To address these problems, we propose a cross-dimensional spatial-temporal feature integration model for LUS video analysis. Specifically, the sliding window and feature difference analysis are first utilized to preprocess the original LUS videos for eliminating invalid and highly similar frames and implementing abstract video. Subsequently, a cross-dimensional feature fusion backbone integrates an improved temporal-C3D network and a self-designed recursive inception-meet-transformer (IMT) network to extract features from different dimensions for fusion. Thereby, comprehensive features can be obtained for characterizing LUS videos. Finally, the Longformer is employed to analyze the temporal dependencies of cross-dimensional features, supplemented by a classification head for evaluating LUS videos. 3018 LUS video clips were collected from 119 patients in three hospitals for the evaluation of the proposed LUS video scoring model. By dividing at the patient level, the training and testing set consist of 2652 clips from 104 patients and 366 clips from 15 patients, respectively. Experimental results of 5-fold cross validation demonstrate that the proposed model achieves outstanding scoring performance, with an accuracy, precision, recall, specificity, F1-score, and AUC of $91.78~\pm ~0.52$ %, $92.19~\pm ~0.76$ %, $91.81~\pm ~0.62$ %, $97.17~\pm ~0.20$ %, $91.94~\pm ~0.44$ %, and $97.83~\pm ~0.19$ %, respectively. The independent testing set also shows the superior generalization capability with a scoring accuracy of $87.65~\pm ~1.12$ %. Moreover, ablation studies confirm that each designed module contributes significantly to the model's performance, and comparative experiments further confirm the superiority of the proposed model compared to previous models. These robust findings highlight the proposed LUS video scoring model's strong potential for clinical deployment.

Offline signature verification remains vulnerable to real-world degradations such as low-resolution scans, transmission artifacts, and environmental noise. We introduce a variant robustness mechanism that foregrounds stroke-level evidence, and instantiate it in a Diffusion-based Robust Signature Verification (DRSV) framework. DRSV integrates two components: (i) a diffusion-driven noise-signature relation module that synthesizes plausible variations to enhance structural resilience, and (ii) a transformer-based verifier that fuses the predicted noise with the signature image via Bi-Directional Channel-Wise Correlation Attention (BCCA) to decide authenticity. Experiments on five public datasets (CEDAR, BHSig–B, BHSig–H, GPDS Synthetic, UTSig) show consistent accuracy gains and improved robustness across diverse perturbations, indicating the effectiveness of the proposed framework.

Recent research on generating 3D assets from single-view inputs by diffusion models has attracted great attention. However, existing diffusion models face challenges such as lack of 3D consistency, high computational costs and rough details in generated content. In this paper, we propose Zero-1-to-3DGS, a novel scheme that balances efficiency and quality in generating high-quality 3D Gaussian models from a single image/text. Specifically, we first utilizes a fine-tuned diffusion model to generate multi-view images consistent with the input image/text. Based on these generated images, we introduce the Gaussian-Based Structural Refiner (GBSRefiner) framework, which is designed to enhance the quality of 3D Gaussian models by leveraging multi-view images. Experimental results show that the Zero-1-to-3DGS outperforms DreamGaussian by 28.2% in the CLIP-Sim metric, and is 6 times faster than SyncDreamer in terms of efficiency. (see https://zero123dgs.github.io/)

Traditional 3D garment creation requires extensive manual operations, resulting in time and labor costs. Recently, 3D Gaussian Splatting has achieved breakthrough progress in 3D scene reconstruction and rendering, attracting widespread attention and opening new pathways for 3D garment reconstruction. However, due to the unstructured and irregular nature of Gaussian primitives, it is difficult to reconstruct high-fidelity, non-watertight 3D garments. In this paper, we present GarmentGS, a dense point cloud-guided method that can reconstruct high-fidelity garment surfaces with high geometric accuracy and generate non-watertight, single-layer meshes. Our method introduces a fast dense point cloud reconstruction module that can complete garment point cloud reconstruction in 10 minutes, compared to traditional methods that require several hours. Furthermore, we use dense point clouds to guide the movement, flattening, and rotation of Gaussian primitives, enabling better distribution on the garment surface to achieve superior rendering effects and geometric accuracy. Through numerical and visual comparisons, our method achieves fast training and real-time rendering while maintaining competitive quality.

Recent mainstream image captioning methods usually adopt two-stage captioners, i.e., calculating the object features of the given image by a pre-trained detector and then feeding them into a language model to generate the descriptive sentences. However, such a two-stage procedure will lead to a task-based information gap that decreases the performance of the captioners, because the object features learned from the detection task are suboptimal representations and cannot provide all the necessary information for subsequent sentence generation. Besides, the object features are usually represented by the last pooling features of the detector that lose the local details of images. In this paper, we propose a novel One-Stage Image Captioner using dynamic multi-sight embedding and alignment, called SeaCap, which directly transforms input images into descriptive sentences in one stage to eliminate the information gap. Specifically, to obtain rich features, we use the Swin Transformer to capture multi-level features, followed by a sights alignment module to alleviate the vision confusion, and then feed them into a novel dynamic multi-sight embedding module to exploit both the global structure and local texture of input images. To enhance the global modeling capacity of the visual encoder, we propose a new dual-dimensional refining module to non-locally model the interaction of the embedded features. As a result, SeaCap can obtain rich and useful information to improve the performance of the captioner. Extensive comparisons on the benchmark MS-COCO, Flickr8K and Flickr30 K datasets verified the superior performance of our method.

Occluded person re-identification (Re-ID) aims to address the potential occlusion problem when matching occluded or holistic pedestrians from different camera views. Many methods use the background as artificial occlusion and rely on attention networks to exclude noisy interference. However, the significant discrepancy between simple background occlusion and realistic occlusion can negatively impact the generalization of the network. To address this issue, we propose a novel transformer-based Attention Disturbance and Dual-Path Constraint Network (ADP) to enhance the generalization of attention networks. Firstly, to imitate real-world obstacles, we introduce an Attention Disturbance Mask (ADM) module that generates an offensive noise, which can distract attention like a realistic occluder, as a more complex form of occlusion. Secondly, to fully exploit these complex occluded images, we develop a DualPath Constraint Module (DPC) that can obtain preferable supervision information from holistic images through dualpath interaction. With our proposed method, the network can effectively circumvent a wide variety of occlusions using the basic ViT baseline. Comprehensive experimental evaluations conducted on person re-ID benchmarks demonstrate the superiority of ADP over state-of-the-art methods.

Chronic obstructive pulmonary disease (COPD) is a common chronic respiratory disease with a high mortality rate. Early diagnosis of risk grading is of great significance for clinical guidance and treatment. In this paper, we propose an automated COPD staging diagnosis model by combining deep/machine learning and computed tomography (CT) scan analysis. This model is composed of three parts: lung parenchyma segmentation, imaging feature analysis, and classification. Firstly, the fast Fourier transform (FFT)-guided dual branch Mamba model is proposed achieve accurate lung parenchyma segmentation in 2-dimensional CT slice images, in which FFT can improve the model’s performance to capture edge and texture information. Next, the Radiomics is employed to analysis the features of segmented 3-dimensional lung parenchyma in CT scans. After effective feature selection, the machine learning classifiers (i.e., K-Nearest Neighbor, Linear Discriminant Analysis, and Support Vector Machine) are used to achieve staged diagnosis of patients with COPD. The experimental results demonstrate that the lung parenchyma segmentation model proposed in this paper has good segmentation performance in CT slice images, with IoU of 0.9819 and Dice of 0.9908. Meanwhile, through 5-fold cross validation, the support vector machine classifier can obtain the best classification performance for COPD staging diagnosis, with the accuracy of 88.57%. The above results prove that the model proposed in this paper has superior diagnosis performance and clinical application potential in small sample situations.

Deep learning based virtual try-on system has achieved some encouraging progress recently, but there still remain several big challenges that need to be solved, such as trying on arbitrary clothes of all types, trying on the clothes from one category to another and generating image-realistic results with few artifacts. To handle this issue, we in this article first collect a new dataset with all types of clothes, i.e., tops, bottoms, and whole clothes, each one has multiple categories with rich information of clothing characteristics such as patterns, logos, and other details. Based on this dataset, we then propose the Arbitrary Virtual Try-On Network (AVTON) that is utilized for all-type clothes, which can synthesize realistic try-on images by preserving and trading off characteristics of the target clothes and the reference person. Our approach includes three modules: (1) Limbs Prediction Module, which is utilized for predicting the human body parts by preserving the characteristics of the reference person. This is especially good for handling cross-category try-on task (e.g., long sleeves ↔ short sleeves or long pants ↔ skirts), where the exposed arms or legs with the skin colors and details can be reasonably predicted; (2) Improved Geometric Matching Module, which is designed to warp clothes according to the geometry of the target person. We improve the TPS based warping method with a compactly supported radial function (Wendland’s Ψ-function); (3) Trade-Off Fusion Module, which is to tradeoff the characteristics of the warped clothes and the reference person. This module is to make the generated try-on images look more natural and realistic based on a fine-tune symmetry of the network structure. Extensive simulations are conducted and our approach can achieve better performance compared with the state-of-the-art virtual try-on methods.

Pneumonia is a contagious disease that poses a great threat to human health. The real-time and free-radiation of lung ultrasound (LUS) makes it an essential tool for diagnosing pneumonia. This paper aims to explore an automatic detection model based on lung ultrasound imaging, called IMT-LUSS, which can achieve effective lung scoring. The model combines inception with transformer, intends to incorporate multi-scale information while considering global information. Firstly, the input is compressed using a stem block. Then it is incorporated into the multi-scale inception meet transformer (IMT) block for information representation, which includes a flexible inception module composed of three convolutional branches with different receptive fields and a pooling branch, and a feature encoding module improved by the multi-head self-attention mechanism with depth-wise convolution. Finally, automatic scoring of LUS images is completed based on feature representation information. 19330 LUS images were employed to verify the proposed IMTLUSS model. Experimental results demonstrate that this model has a great LUS scoring performance with high accuracy of 99.44 ± 0.14%. Meanwhile, the ablation experiments on structure and comparative experiments with other models proved its significant superiority, indicating the potential in future clinical applications.

Recent advances in single image super-resolution (SISR) have achieved remarkable performance through deep learning. However, the high computational cost hinders the deployment of SISR models on edge devices. Instead of proposing new SISR models, a new trend is emerging to improve network efficiency by reducing parameters, FLOPs, and inference time through slight modifications to the original models. However, recent methods usually focus on reducing only one of three metrics, i.e., FLOPs, parameters and inference time, which inevitably increases the other two metrics. In this paper, we propose a novel Adaptive Student Inference Network (ASIN) on popular SISR models, which aims at reducing FLOPs and inference time while maintaining the number of parameters and restoring clearer high-resolution images. Specifically, our ASIN divides a SISR model into three components (head, body and tail) and adopts various strategies for each part. For head and tail parts, to ensure the restored images contain more detailed information, a novel auxiliary Enhanced Teacher Network (ETNet) is designed, which is trained with the ground-truth images to obtain more prior knowledge to guide student network to extract more accurate textures using a new knowledge distillation method. For the body part, owing to the varying difficulties of the reconstructions in different regions, we propose an Adaptive Depth Predicted Module (ADPM) to dynamically shorten average depth of network to reduce the computational cost of overall network. Extensive experiments on two datasets demonstrate the effectiveness and state-of-the-art performance of our ASIN compared to its counterparts.

Clothes attribution prediction is the key technology for users to automatically describe clothing characteristics. Most current methods are first to detect the multiple clothes, and then crop out the clothes and feed to a certain network for clothes attribution prediction. But this two-stage approach is time- and resource- consuming; on the other hand, one-stage approach can provide an effective and efficient solution by integrating clothes detection and attribution prediction into an end-to-end framework. But the one-stage approach tends to explore anchor-based detectors causing high sensitivity to the hyperparameters and high computational complexity from dense anchors. In addition, it may also confront with optimization contradiction problem in the training procedure, as the clothes detection and attribution prediction branches demand diverse optimization. In this work, to handle the above problems, we aim to develop an end-to-end anchor-free framework by involving an additional branch for joint clothes detection and attribution prediction. To handle the optimization contradiction in two branches, we encode the backbone feature map as pixel-level dense queries and decode them via deformable transformer as the output features that are fed into detection and prediction branches, respectively. In this way, the features of detection and prediction branches can be decoupled and the optimization contradiction can be naturally solved. To further enhance the prediction accuracy, we in the prediction branch also develop a special attention strategy and loss function to adaptively integrate the peer attribution relationships into feature learning as well as to avoid mutual suppression for hierarchical attributions. Extensive simulation results verify the effectiveness of the proposed work.

Image captioning is still a challenging task aiming at describing the contents of image by words. Current image caption methods usually assume the object relation to be important if the semantic and spatial geometric relationships between objects are close and large, but the relations meeting this assumption are not necessarily important to describe the contents of image in a fine-grained way. That is, the importance of fine-grained object relations is not properly taken into account. Besides, current Transformer based image caption models also fail to consider the importance of fine-grained objects, since they generate all the words of a sentence at one time, which cannot Figure out which objects are more important and vice versa. In this paper, we propose a novel Fine-grained Adaptive Object Transformer (FineFormer) network, which can jointly discover the importance of fine-grained objects and object relations for image captioning. Specifically, a new concept of adaptive soft-foreground attention is proposed to highlight the fine-grained objects dominating the descriptive contents. To characterize and calculate the important relations between fine-grained objects, we also propose an adaptive object relation attention to refine the object relation from the generation process of relation. As such, FineFormer can describe the contents of image more accurately, by reducing the interference of unimportant objects in the background. Extensive experiments on the highly-competitive MS-COCO dataset demonstrated the superiority of our FineFormer.

Clothes image search is the key technique to effectively search the clothes items that are most relevant to the query clothes given by the customer. In this work, we propose an Anchor-free framework for clothes image search by adopting an additional Re-ID branch for similarity learning and global mask branch for instance segmentation. The Re-ID branch is to extract richer feature of target clothes, where we develop a mask pooling layer to aggregate the feature by utilizing the mask of target clothes as the guidance. In this way, the extracted feature will involve more information covered by the mask area of targets instead of only the center point; the global mask branch is to be trained with detection and Re-ID branches simultaneously, where the estimated mask of target clothes can be utilized in reference procedure to guide the feature extraction. Finally, to further enhance the performance of retrieval, we have introduced a match loss to further fine-tune the Re-ID embedding branch in the framework, so that the clothes target can be closer to the same one, while be farther away from different clothes targets. Extensive simulations have been conducted and the results verify the effectiveness of the proposed work.

Convolutional Neural Networks (CNNs) are powerful for image representation, but the convolution operation may be influenced and degraded by the included noise, and the deep features may not be fully learned. In this paper, we propose a new encoder-decoder based image restoration network, termed Robust Low-Rank Convolution Network with Feature Denoising (LRCnet). LRCnet presents a novel low-rank convolution (LR-Conv) for image representation, and a residual dense connection (RDC) for feature fusion between encoding and decoding. Different from directly splitting convolution into ordinary convolution and mirror convolution as existing work, LR-Conv deploys a feature denoising module after the ordinary convolution to remove noise for mirror convolution. A low-rank embedding process is then used to project the convolutional features into a robust low-rank subspace, which can retain the local geometry of input signal to some extent and separate the signal and noise by finding low-rank structure of features to reduce the impact of noise on convolution. Besides, most networks increase the depth of network simply to obtain deep information and lack of effective connections to fuse the multilevel features, which may not fully discover the deep features in various layers. Thus, we design a residual dense connection with a channel attention to connect multilevel feature effectively to obtain more useful information to enhance the data representation. Extensive experiments on several datasets verified the effectiveness of LRCnet for image denoising.

The performance of spectral clustering heavily relies on the quality of affinity matrix. A variety of affinity-matrix-construction (AMC) methods have been proposed but they have hyperparameters to determine beforehand, which requires strong experience and leads to difficulty in real applications, especially when the inter-cluster similarity is high and/or the dataset is large. In addition, we often need to choose different AMC methods for different datasets, which still depends on experience. To solve these two challenging problems, in this paper, we present a simple yet effective method for automated spectral clustering. First, we propose to find the most reliable affinity matrix via grid search or Bayesian optimization among a set of candidates given by different AMC methods with different hyperparameters, where the reliability is quantified by the \textit{relative-eigen-gap} of graph Laplacian introduced in this paper. Second, we propose a fast and accurate AMC method based on least squares representation and thresholding and prove its effectiveness theoretically. Finally, we provide a large-scale extension for the automated spectral clustering method, of which the time complexity is linear with the number of data points. Extensive experiments of natural image clustering show that our method is more versatile, accurate, and efficient than baseline methods.

While three-dimensional (3D) late gadolinium-enhanced (LGE) magnetic resonance (MR) imaging provides good conspicuity of small myocardial lesions with short acquisition time, it poses a challenge for image analysis as a large number of axial images are required to be segmented. We developed a fully automatic convolutional neural network (CNN) called cascaded triplanar autoencoder M-Net (CTAEM-Net) to segment myocardial scar from 3D LGE MRI. Two sub-networks were cascaded to segment the left ventricle (LV) myocardium and then the scar within the pre-segmented LV myocardium. Each sub-network contains three autoencoder M-Nets (AEM-Nets) segmenting the axial, sagittal and coronal slices of the 3D LGE MR image, with the final segmentation determined by voting. The AEM-Net integrates three features: (1) multi-scale inputs, (2) deep supervision and (3) multi-tasking. The multi-scale inputs allow consideration of the global and local features in segmentation. Deep supervision provides direct supervision to deeper layers and facilitates CNN convergence. Multi-task learning reduces segmentation overfitting by acquiring additional information from autoencoder reconstruction, a task closely related to segmentation. The framework provides an accuracy of 86.43% and 90.18% for LV myocardium and scar segmentation, respectively, which are the highest among existing methods to our knowledge. The time required for CTAEM-Net to segment LV myocardium and the scar was 49.72 ± 9.69s and 120.25 ± 23.18s per MR volume, respectively. The accuracy and efficiency afforded by CTAEM-Net will make possible future large population studies. The generalizability of the framework was also demonstrated by its competitive performance in two publicly available datasets of different imaging modalities.

Deep neural network (DNN) compression can reduce the memory footprint of deep networks effectively, so that the deep model can be deployed on the portable devices. However, most of the existing model compression methods cost lots of time, e.g., vector quantization or pruning, which makes them inept to the real-world applications that need fast online computation. In this paper, we therefore explore how to accelerate the model compression process by reducing the computation cost. Then, we propose a new deep model compression method, termed Dictionary Pair-based Data-Free Fast DNN Compression, which aims at reducing the memory consumption of DNNs without extra training and can greatly improve the compression efficiency. Specifically, our proposed method performs tensor decomposition on the DNN model with a fast dictionary pair learning-based reconstruction approach, which can be deployed on different layers (e.g., convolution and fully-connection layers). Given a pre-trained DNN model, we first divide the parameters (i.e., weights) of each layer into a series of partitions for dictionary pair-based fast reconstruction, which can potentially discover more fine-grained information and provide the possibility for parallel model compression. Then, dictionaries of less memory occupation are learned to reconstruct the weights. Extensive experiments on popular DNNs (i.e., VGG-16, ResNet-18 and ResNet-50) showed that our proposed weight compression method can significantly reduce the memory footprint and speed up the compression process, with less performance loss.

Real-world data in emerging applications may suffer from highly-skewed class imbalanced distribution, however how to deal with this kind of problem appropriately through deep learning needs further investigation. In this paper, we mainly propose a novel cross-entropy based loss function, referred to as Additive Scale Loss (ASL), for deep representation learning and imbalanced classification. To deal with the class imbalanced problem, ASL aims at increasing the loss in case of misclassification, which can avoid the superimposed loss values caused by the large amount of easily classified data in the unbalanced database to dominate the loss value of misclassified data. Moreover, in real-world applications, one data source may be used for multiple scenarios, such as classification and embedding learning, however training two separable models to handle these problems is costly, especially in deep learning area. To tackle this issue, we present and integrate a discriminative inter-class separation term into ASL, and propose a discriminative ASL (D-ASL), which can not only improve the classification performance, but also obtain discriminative representations simultaneously. The discriminative inter-class separation term is general, and can be easily integrated to other loss functions, such as CE and FL, as the byproducts. Finally, a new deep convolutional neural network equipped with D-ASL and a fully-connected (FC) layer is proposed, which can classify the imbalanced image data and obtain the discriminative representations at the same time. Extensive experimental results verified the superior performance of our method.

Although supervised single image deraining (SID) have obtained impressive results, they still cannot obtain satisfactory results on real images for the weak generalization of rain removal capacity. In this paper, we mainly discuss the semi-supervised SID and propose a new GAN-based deraining network called Semi-DerainGAN, which can use both synthetic and real data in a uniform network based on two supervised and unsupervised processes. For this task, a semi-supervised rain streak learner termed SSRML sharing the same parameters of both processes is derived, which makes the real images contribute more rain streak information, so that the resulted model has a strong generalization power to the real SID task. We also contribute a new real-world rain image dataset called Real200 to alleviate the difference between both synthetic and real image domains. Extensive results on public datasets show that our model can obtain competitive results, especially on the real rain images.

Learning and extracting high-level features from point cloud is the key to improving the segmentation performances on point clouds for many networks. At present, many networks present very deep structures to extract high-level features for 3D perception. However, we argue that even better results can be achieved by (i) building feature vectors that integrates multi-scale geometric features, and (ii) exerting discriminative constraints on the learning of mid-levels features. In this paper, we propose a Multi-scale Neighborhood Feature Extraction and Aggregation Model (MNFEAM) to enhance feature extraction for point cloud learning. We try to first extract multi-scale neighborhood information for each input point and then aggregate local information of a mid-level locality feature space, and finally integrate the aggregated local and the global feature vectors. A new discriminative loss function is designed to strengthen the coarse semantics on mid-levels features so that the semantic abstraction process can be improved and accelerated. We improve the performances of three popular networks for point cloud segmentation using the proposed MNFEAM on standard 3D datasets.

In this paper, we technically propose a novel framework called Deep Self-representative Concept Factorization Network (DSCF-Net), for clustering deep features. To improve the representation and clustering abilities, DSCF-Net explicitly considers discovering hidden deep semantic features, enhancing the robustness properties of the deep factorization to noise and preserving the local manifold structures of deep features. Specifically, DSCF-Net integrates the robust deep concept factorization, deep self-expressive representation and adaptive locality preserving feature learning into a unified framework. To discover hidden deep representations, DSCF-Net designs a hierarchical factorization architecture using multiple layers of linear transformations, where the hierarchical representation is performed by formulating the problem as optimizing the basis concepts in each layer to improve the representation indirectly. DSCF-Net also improves robustness by subspace recovery for sparse error correction firstly and then performs deep factorization in the recovered visual subspace. To obtain localitypreserving representations, we also present an adaptive deep self-representative weighting strategy by using the coefficient matrix as adaptive weights to keep the locality of representations. Extensive results show that DSCF-Net delivers state-of-the-art performance on several public databases.

Classical matrix completion methods are not effective in recovering missing entries of data drawn from multiple subspaces because the matrices are often of high-rank. Recently a few advanced matrix completion methods were proposed to solve the problem but they are not scalable to large matrices and big data problems. This paper proposes a sparse factorization method for matrix completion on multiple-subspace data. The method factorizes the given incomplete matrix into a dense matrix and a sparse matrix, while the factorization errors of the observed entries are minimized. To solve the optimization problem, an accelerated proximal alternating linearized minimization (APALM) algorithm is proposed. As a non-trivial task owing to the alternation, linearization, nonconvexity, and extrapolation, the convergence of APALM is proved. APALM can solve a large class of optimization problems such as matrix factorization with nonsmooth regularizations. In addition, we show that, to recover an m × n matrix consisting of data drawn from k subspaces of dimension r0, the number of observed entries required in our matrix completion method is O(nr0logklog n) while that in conventional methods is O(nr0klog n), which theoretically proves the superiority of our method on multiple-subspace data and high-rank matrices. The proposed matrix completion method is compared with state-of-the-art on synthetic data and real collaborative filtering problems. The experimental results corroborate that the proposed method can handle large matrices efficiently and provide high recovery accuracy.

Recommender systems are currently utilized widely in e-commerce for product recommendations and within content delivery platforms. Previous studies usually use independent features to represent item content. As a result, the relationship hidden among the content features is overlooked. In fact, the reason that an item attracts a user may be attributed to only a few set of features. In addition, these features are often semantically coupled. In this paper, we present an optimization model for extracting the relationship hidden in content features by considering user preferences. The learned feature relationship matrix is then applied to address the cold-start recommendations and content-based recommendations. It could also easily be employed for the visualization of feature relation graphs. Our proposed method was examined on three public datasets: 1) hetrec-movielens-2k-v2; 2) book-crossing; and 3) Netflix. The experimental results demonstrated the effectiveness of our method in comparison to the state-of-the-art recommendation methods.

Most existing models of energy efficiency retrofit are able to evaluate energy saving and retrofit cost for a certain stakeholder, but unable to guide how to allocate retrofit task and incentive among multiple stakeholders. The multistakeholder situation is firstly modeled in the proposed crowdsourcing model (CM), which contributes to quantify the utility of each competitive stakeholder with respect to participation decision. To solve the CM, a Stackelberg game approach is newly developed in this article to find rational and efficient strategies of task/incentive allocation. For the building energy efficiency retrofit, the challenge of CM is to handle mixed-integer decisions of energy service companies. We prove the existence of Stackelberg equilibrium (SE), which introduces the optimal budget, and the Nash equilibrium of task allocation. To compute the SE of CM, effective search algorithms are designed based on best response and optimization techniques. Simulation results have verified the CM and game theoretical approach. The resulted SE has provided stable and efficient strategies of incentive/task allocation.

Change detection has received extensive attention because of its realistic significance and broad application fields. However, none of the existing change detection algorithms can handle all scenarios and tasks so far. Different from the most of contributions from the research community in recent years, this paper does not work on designing new change detection algorithms. We, instead, solve the problem from another perspective by enhancing the raw detection results after change detection. As a result, the proposed method is applicable to various kinds of change detection methods, and regardless of how the results are detected. In this paper, we propose Fast Spatiotemporal Tree Filter (FSTF), a purely unsupervised detection method, to enhance coarse binary detection masks obtained by different kinds of change detection methods. In detail, the proposed FSTF has adopted a volumetric structure to effectively synthesize spatiotemporal information of the same target from the current time and history frames to enhance detection. The computational complexity analyzed in the view of graph theory also show that the fast realization of FSTF is a linear time algorithm, which is capable of handling efficient on-line detection tasks. Finally, comprehensive experiments based on qualitative and quantitative analysis verify that FSTF-based change detection enhancement is superior to several other state-of-the-art methods including fully connected Conditional Random Field (CRF), joint bilateral filter, and guided filter. It is illustrated that FSTF is versatile enough to also improve saliency detection as well as semantic image segmentation.

The graph-based semisupervised label propagation (LP) algorithm has delivered impressive classification results. However, the estimated soft labels typically contain mixed signs and noise, which cause inaccurate predictions due to the lack of suitable constraints. Moreover, the available methods typically calculate the weights and estimate the labels in the original input space, which typically contains noise and corruption. Thus, the encoded similarities and manifold smoothness may be inaccurate for label estimation. In this article, we present effective schemes for resolving these issues and propose a novel and robust semisupervised classification algorithm, namely the triple matrix recovery-based robust auto-weighted label propagation framework (ALP-TMR). Our ALP-TMR introduces a TMR mechanism to remove noise or mixed signs from the estimated soft labels and improve the robustness to noise and outliers in the steps of assigning weights and predicting the labels simultaneously. Our method can jointly recover the underlying clean data, clean labels, and clean weighting spaces by decomposing the original data, predicted soft labels, or weights into a clean part plus an error part by fitting noise. In addition, ALP-TMR integrates the auto-weighting process by minimizing the reconstruction errors over the recovered clean data and clean soft labels, which can encode the weights more accurately to improve both data representation and classification. By classifying samples in the recovered clean label and weight spaces, one can potentially improve the label prediction results. Extensive simulations verified the effectivenss of our ALP-TMR.

In this paper, we propose a novel Convolutional Neural Network (CNN) structure for general-purpose multi-task learning (MTL), which enables automatic feature fusing at every layer from different tasks. This is in contrast with the most widely used MTL CNN structures which empirically or heuristically share features on some specific layers (e.g., share all the features except the last convolutional layer). The proposed layerwise feature fusing scheme is formulated by combining existing CNN components in a novel way, with clear mathematical interpretability as discriminative dimensionality reduction, which is referred to as Neural Discriminative Dimensionality Reduction (NDDR). Specifically, we first concatenate features with the same spatial resolution from different tasks according to their channel dimension. Then, we show that the discriminative dimensionality reduction can be fulfilled by 1×1 Convolution, Batch Normalization, and Weight Decay in one CNN. The use of existing CNN components ensures the end-to-end training and the extensibility of the proposed NDDR layer to various state-of-the-art CNN architectures in a "plug-and-play" manner. The detailed ablation analysis shows that the proposed NDDR layer is easy to train and also robust to different hyperparameters. Experiments on different task sets with various base network architectures demonstrate the promising performance and desirable generalizability of our proposed method. The code of our paper is available at https://github.com/ethanygao/NDDR-CNN.

Recently, technological advances have made possible the measure of daily (even hourly) user(sales) growth of a product or service. Here, questions comes: how users grow and how the promotion influence the user growth? Here, we develop a model that can describe the growth of the user population of a newly launched product or service, which can answer this question. To develop this model, we consider a network of interacting individuals, whose actions or transitions are determined by the states (behaviour) of their neighbours as well as their own personal decisions. This model leads to a simple growth equation connecting the growth of user population with the total number of prospective users, and the effects of peer influence and personal choice. Several real-life datasets of a variety of products and services have been analyzed. Results suggest that they all follow the proposed growth equation. The numerical procedure for finding the model parameters thus ensure the relative effectiveness, market size and promotional efforts to be estimated from the available historical growth data.

Vision-based techniques are gradually being applied to civil engineering fields by aiding human resources to inspect and assess the condition of structures. This paper presents a vision-based inspection system to detect bends of the bridge chords in UAV images. The main parts of this paper firstly present a novel region proposal algorithm to localize bridge chord bends without distinct boundaries and then present a fusion-based deep semantic segmentation network that improves the detection performance by fusing spatial information from neighborhood. The proposed system achieves approximate 0.7762 Mean IOU and 0.6424 BF Score on the field dataset, which outperforms the other state-of-the-art systems.

During the past decade, graph-based semi-supervised learning has become one of the most important research areas in machine learning and artificial intelligence community. In this paper, we propose a sub-graph to construct the graph for semi-supervised learning (SSL). The new graph is scalable so that it can be extended to large-scale data. Based on this graph, we then propose a sub-graph regularization for scalable SSL. It can also project the new-coming data to infer its label for handling out-of-sample problem. Simulation results show that the proposed method can achieve better performance compared with other state-of-the-art graph based SSL methods.

The study of patient behaviours (vital sign, physical action and emotion) is crucial to improve one’s quality of life. The only solution for handling and managing millions of people’s behaviours and health would be big data and IoT technology because most of the countries are lack of medical professionals. In this paper, a big data and IoT-based patient behaviour monitoring system have proposed. Qualitative studies are carried out on the selected behaviours analytics, cardiovascular disease identification and fall detection. At last, authors have summarised the general challenges like trust, privacy, security and interoperability as well as special challenges in various sectors: government, legislators, research institutions, information technology companies and patients.

Kernel methods have been successfully applied to the areas of pattern recognition and data mining. In this paper, we mainly discuss the issue of propagating labels in kernel space. A Kernel-Induced Label Propagation (Kernel-LP) framework by mapping is proposed for high-dimensional data classification using the most informative patterns of data in kernel space. The essence of Kernel-LP is to perform joint label propagation and adaptive weight learning in a transformed kernel space. That is, our Kernel-LP changes the task of label propagation from the commonly-used Euclidean space in most existing work to kernel space. The motivation of our Kernel-LP to propagate labels and learn the adaptive weights jointly by the assumption of an inner product space of inputs, i.e., the original linearly inseparable inputs may be mapped to be separable in kernel space. Kernel-LP is based on existing positive and negative LP model, i.e., the effects of negative label information are integrated to improve the label prediction power. Also, Kernel-LP performs adaptive weight construction over the same kernel space, so it can avoid the tricky process of choosing the optimal neighborhood size suffered in traditional criteria. Two novel and efficient out-of-sample approaches for our Kernel-LP to involve new test data are also presented, i.e., (1) direct kernel mapping and (2) kernel mapping-induced label reconstruction, both of which purely depend on the kernel matrix between training set and testing set. Owing to the kernel trick, our algorithms will be applicable to handle the high-dimensional real data. Extensive results on real datasets demonstrate the effectiveness of our approach.

In this paper, we investigate the robust transductive label prediction problem. Technically, a Robust Adaptive Label Propagation framework by Double Matrix Decomposition, called ALP-MD, is proposed for the semi-supervised data classification. Compared with existing transductive label propagation models, our ALP-MD improves the classification power by performing label prediction in the clean data space and clean label space at the same time. More specifically, our ALP-MD clearly integrates the idea of double matrix decomposition into the process of label prediction for the noise removal. Since the predicted soft labels usually contains noise and mixed signs, our ALP-MD explicitly decomposes the predicted soft label matrix into a clean soft label matrix and a noise term and then estimates the hard label based on the clean soft label matrix for more accurate classification. In addition, ALP-MD also involves a regularization term to model the noise in data, integrates the adaptive weights learning into the process of robust label prediction and moreover performs the weights learning in the clean data space. Thus, our ALP-MD can explicitly ensure the learned weights to be informative as much as possible and to be joint optimal for both representation and classification, and potentially enhance the label prediction ability. Extensive comparisons demonstrated its effectiveness.

Abstract Controlling stroke size in Fast Style Transfer remains a difficult task. So far, only a few attempts have been made towards it, and they still exhibit several deficiencies regarding efficiency, flexibility, and diversity. In this paper, we aim to tackle these problems and propose a recurrent convolutional neural subnetwork, which we call recurrent stroke‐pyramid , to control the stroke size in Fast Style Transfer. Compared to the state‐of‐the‐art methods, our method not only achieves competitive results with much fewer parameters but provides more flexibility and efficiency for generalizing to unseen larger stroke size and being able to produce a wide range of stroke sizes with only one residual unit. We further embed the recurrent stroke‐pyramid into the Multi‐Styles and the Arbitrary‐Style models, achieving both style and stroke‐size control in an entirely feed‐forward manner with two novel run‐time control strategies.

Books, as a representative of lengthy documents, convey rich semantics. Traditional document modeling methods, such as bag-of-words models, have difficulty capturing such rich semantics when only considering term-frequency features. In order to explore term spatial distributions over a book, a tree-structured book representation is investigated in this paper. Moreover, an efficient learning framework, Tree2Vector, is introduced for mapping tree-structured book data into vectorial space. In particular, we present two types of locality reconstruction (LR) models: Euclidean-type and cosine-type, during the transformation process of tree structures into vectorial representations. The LR is used for modeling the reconstruction process, in which each parent node in a tree is supposed to be reconstructed by its child nodes. The prominent advantage of this Tree2Vector framework is that it solely utilizes the local information within a single book tree. In addition, extensive experimental results demonstrate that Tree2Vector is able to deliver comparable or better performance in comparison to methods that consider the information of all trees in a database globally. Experimental results also suggest that cosine-type LR consistently performs better than Euclidean-type LR in applications of book and author recommendations.

In our analysis, we consider daily rainfall records y(n) from 3825 areas covering entire China. The daily rainfall records is 50 years long from 1961 to 2011. We study the statistical properties of the daily rainfall data and return intervals Tqbetween two consecutive rainfall records above some threshold q. Using the detrended fluctuation analysis (DFA) method to analyze the long-term correlation properties of the rainfall record y(n) and return intervals Tq, we find that the long-term correlation is not exist in rainfall record, while a weak long-term correlation is found in return intervals, pointing to a random regulation of rainfall dynamics. Then, we further analyze the probability distribution function (PDF) of the rainfall record and return intervals of all the 3825 areas, and find that all the PDF obey a power-law distribution.

Rolling element bearings play an important role in ensuring the availability of industrial machines. Unexpected bearing failures in such machines during field operation can lead to machine breakdown, which may have some pretty severe implications. However, the insufficiency of labeled samples is major problem for handling fault diagnosis problem. To address such concern, we propose a semi-supervised method for diagnosing faulty bearings by utilizing unlabeled samples. The superiority of our algorithm has been validated by comparison with other state-of art methods based on a rolling element bearing data. The classification accuracy of bearing data show that our algorithm is able to recognize different bearing fault categories effectively. Thus, it can be considered as a promising method for fault diagnosis.

During the past decade, graph-based semi-supervised learning has become one of the most important research areas in machine learning and artificial intelligence community. In this paper, we propose a Capped l2,1-Norm Regularized Dictionary Learning to construct the graph for semi-supervised learning (SSL). The new sparse coding is robust to the outliers and insensitive to noise. Then, the developed graph construction can better characterize the geometrical structure of data mainfold and be utilized for efficient graph based SSL. the proposed SSL methods can also be easily extended to deal with out-of-sample data. Simulation results show that the proposed method can achieve better performance compared with other state-of-the-art graph based SSL methods.

This paper investigates the salient machine faults representation-based classification issue by dictionary learning. A novel structured latent label consistent dictionary learning (LLC-DL) model is proposed for joint discriminative salient representation and classification. Our LLC-DL deals with the tasks by solving one objective function that aims to minimize the structured reconstruction error, structured discriminative sparse-code error and classification error simultaneously. Also, LLC-DL decomposes given signals into a sparse reconstruction part over structured latent weighted discriminative dictionary, a salient feature extraction part and an error part fitting noise. Specifically, the dictionary is learnt atom by atom, where each dictionary atom is learnt with a latent vector that reduces the disturbance between interclass atoms. The structured coding coefficients are calculated via minimizing the reconstruction error and discriminative sparse code error simultaneously. The salient representations are learnt by embedding signals onto a projection and a robust linear classifier is then trained over the learned salient features directly so that features can be ensured to be optimal for classification, where robust l2,1-norm imposed on the classifier can make the prediction results more accurate. By including a salient feature extraction term, the classification approach of LLC-DL is very efficient, since there is no need to involve an extra time-consuming sparse reconstruction process with the well-trained dictionary for each test signal. Extensive simulations versify the effectiveness of our algorithm.

This paper focuses on an important and fundamental problem of connecting two points by a cable, subject to a tradeoff between cost and earthquake survivability. In particular, we address the problem of selecting a route for laying a cable under arbitrary topography, based on earthquake data. First, we derive a semi-supervised probability density estimation model for the likelihood of earthquake disaster. Based on this probabilistic model, we generate a nearest neighbor graph. The graph represents each data point with a four-dimensional space formed by the three-dimensional undersea coordinates and the one-dimensional data of earthquake disaster level. It then forms the weight on graph between any positions. The data used in this study are all real data of undersea topography and earthquake information of the Taiwan Strait. As a result, both the undersea topology and the earthquake level can be transferred into a distance for shortest route finding. Finally, Dijkstra's algorithm is used for finding the optimal shortest route for cabling between the two given points on the graph. Extensive simulations based on a synthetic dataset and the Taiwan Strait real-world dataset corroborate the effectiveness of the proposed method.

We propose two nuclear- and L2,1-norm regularized 2D neighborhood preserving projection (2DNPP) methods for extracting representative 2D image features. 2DNPP extracts neighborhood preserving features by minimizing a Frobenius norm-based reconstruction error that is very sensitive noise and outliers in given data. To make the distance metric more reliable and robust, and encode the neighborhood reconstruction error more accurately, we minimize the nuclear- and L2,1-norm-based reconstruction error, respectively and measure it over each image. Technically, we propose two enhanced variants of 2DNPP, nuclear-norm-based 2DNPP and sparse reconstruction-based 2DNPP. Besides, to optimize the projection for more promising feature extraction, we also add the nuclear- and sparse L2,1-norm constraints on it accordingly, where L2,1-norm ensures the projection to be sparse in rows so that discriminative features are learnt in the latent subspace and the nuclear-norm ensures the low-rank property of features by projecting data into their respective subspaces. By fully considering the neighborhood preserving power, using more reliable and robust distance metric, and imposing the low-rank or sparse constraints on projections at the same time, our methods can outperform related state-of-the-arts in a variety of simulation settings.

A multi-context mechanism is newly reported to solve large-scale optimization problems (separable and nonseparable) within a general cooperative co-evolution (CC) framework. The basic CC is widely used to decompose a large-scale problem into several less difficult subproblems. When any two subproblems have no interaction, for example, when the problem is separable, the basic CC is effective. However, in practical cases there exist intensive interactions between subproblems, then the basic CC fails to find the global optimum. In this paper, the main reason of such failures has been studied and summarized. A general CC is proposed to use multiple context variables to avoid trapping caused by interactions. For the 500-dimension Rosenbrock's function with the optimum 0, the best result reported in existing CC methods is at the 102level, but the global optimum can be found in the proposed CC. On a comprehensive set of benchmark, the proposed CC performs significantly better than existing CC in terms of accuracy.

We propose a new transductive label propagation method, termed Adaptive Neighborhood Propagation (Adaptive-NP) by joint L2,1-norm regularized sparse coding, for semi-supervised classification. To make the predicted soft labels more accurate for predicting the labels of samples and to avoid the tricky process of choosing the optimal neighborhood size or kernel width for graph construction, Adaptive-NP seamlessly integrates sparse coding and neighborhood propagation into a unified framework. That is, the sparse reconstruction error and classification error are combined for joint minimization, which clearly differs from traditional methods that explicitly separate graph construction and label propagation into independent steps, which may result in inaccurate predictions. Note that our Adaptive-NP alternately optimize the sparse codes and soft labels matrices, where the sparse codes are used as adaptive weights for neighborhood propagation at each iteration, so the tricky process of determining neighborhood size or kernel width is avoided. Besides, for enhancing sparse coding, we use the L2,1-norm constraint on the sparse coding coefficients and the reconstruction error at the same time for delivering more accurate and robust representations. Extensive simulations show that our model can deliver state-of-the-art performances on several public datasets for classification.

We propose a nuclear-norm regularized two-dimensional neighborhood preserving projection (2DNPP) for extracting representative 2D image features. Note that 2DNPP extracts neighborhood preserving features through minimizing the reconstruction error, but the Frobenius norm based metric is sensitive to noise and outliers. To make the distance metric more reliable and model the neighborhood reconstruction error more accurately, we impose the nuclear-norm on the neighborhood reconstruction error and measure it over each image. Technically, we propose a new variant of 2DNPP termed nuclear-norm based 2DNPP (N-2DNPP). Besides, to make delivered projection promising for feature extraction, we also include the nuclear-norm constraint on projection accordingly, where the low-rank projection can embed data into their respective subspaces. Our method can outperform related state-of-the-arts in a variety of simulation settings.

In most sparse coding based subspace clustering problems, using the non-convex lp-norm minimization (01-norm minimization. In this paper, we propose a sparse subspace clustering via joint lp-norm and l2,p-norm minimization, where the lp-norm imposed on sparse representations can achieve more sparsity for clustering while l2,p-norm imposed on reconstructed error can handle outlier pursuit. We also propose an iterative solution to solve the proposed problem based on Iterative Shrinkage/Thresholding (IST) method. In addition, to the best knowledge, utilizing IST for solving l2,p-norm minimization problem can be the first work in our paper and there is no such work before. Finally, to demonstrate the improved performance of the proposed method, comparative study was performed on benchmark problems of image clustering. Thoroughly experimental studies on real world datasets show that the method can significantly outperform other state-of-the-art methods.

We propose a new discriminative dictionary learning framework termed Latent Label Consistent K-SVD (LLC-KSVD) for representing and classifying machine faults. Our LLC-KSVD handles the task by minimizing the reconstruction, discriminative sparse-code and classification errors at the same time. To enhance the representation and classification powers, LLC-KSVD aim to decompose given data into a sparse reconstruction part, a salient feature part and an error part. The salient features are learnt by embedding data onto a projection and a classifier is then trained over extracted salient features so that features are ensured to be optimal for classification. Thus, the classification approach of our LLC-KSVD is very efficient, since there is no need to involve a time-consuming sparse reconstruction process with well-trained dictionary for each test signal as existing models. Besides, to make the classifier be robust to noise and outliers, we regularize the l2,¿-norm on classifier so that the predictions are more accurate. Simulations on several machine fault datasets demonstrate the state-of-the-art performance by our LLC-KSVD.

Logistics network optimization plays a critical role in contemporary logistics planning and supply chain network designs, and the vehicle route planning is essential for logistics network optimizations. In this paper, we present an efficient and effective approach for vehicle route planning. The new approach has utilized multiple spanning trees as a criterion to categorize the customers into several sub areas. In addition, by iterative choose some customers in the boundary of sub areas and resign them to different spanning trees, we can get a more compact ones with smaller distances. As a result, since the spanning tree represent the lower bound of vehicle route in all sub areas, we can also conduct the vehicular dispatching in each sub areas. Extensive simulation has verified the effectiveness of the proposed methods.

In this paper, we propose a semisupervised label consistent dictionary learning (SSDL) framework for machine fault classification. SSDL is a semisupervised extension of recent fully supervised label consistent dictionary learning approach, since the number of labeled machine data is usually limited in practice. To enable the supervised dictionary learning model to use both labeled and commonly readily available unlabeled data for enhancing performance, we propose to incorporate the merits of label prediction and present a joint label consistent dictionary learning and adaptive label prediction technique. In this setting, we first employ the existing label prediction model to estimate the labels of unlabeled training signals in a transductive fashion for enriching supervised prior. Then, we use predicted labeled data for label consistent dictionary learning. After that, we apply the discriminant sparse codes as the adaptive reconstruction weights for label prediction to update the estimated labels of unlabeled training data and the discriminative sparse codes matrix for label consistent dictionary learning so that classification performance can be enhanced. Thus, an informative dictionary, a sparse-code matrix, and an optimal multiclass classifier can be alternately obtained from one objective function. Besides, the tricky process of choosing optimal kernel width and neighborhood size can also be effectively voided in our scheme due to the adaptive weights. Extensive simulations on several machine fault datasets show that our SSDL method can deliver enhanced performance over other state-of-the-arts for machine fault classification.

Recovering low-rank and sparse subspaces jointly for enhanced robust representation and classification is discussed. Technically, we first propose a transductive low-rank and sparse principal feature coding (LSPFC) formulation that decomposes given data into a component part that encodes low-rank sparse principal features and a noise-fitting error part. To well handle the outside data, we then present an inductive LSPFC (I-LSPFC). I-LSPFC incorporates embedded low-rank and sparse principal features by a projection into one problem for direct minimization, so that the projection can effectively map both inside and outside data into the underlying subspaces to learn more powerful and informative features for representation. To ensure that the learned features by I-LSPFC are optimal for classification, we further combine the classification error with the feature coding error to form a unified model, discriminative LSPFC (D-LSPFC), to boost performance. The model of D-LSPFC seamlessly integrates feature coding and discriminative classification, so the representation and classification powers can be enhanced. The proposed approaches are more general, and several recent existing low-rank or sparse coding algorithms can be embedded into our problems as special cases. Visual and numerical results demonstrate the effectiveness of our methods for representation and classification.

In this paper, we mainly present a Semi-Supervised Label Consistent KSVD (S2KSVD) algorithm for representing and classifying machine faults. The formulation of our S2KSVD is an improvement to the recent label consistent K-SVD (LC-KSVD), because LC-KSVD is a fully supervised approach, and needs to use supervised class information of all training data to compute a reconstructive & discriminative dictionary. But labeled signals are often expensive to obtain, while in contrast unlabeled signals can be easily captured with low expense from the real world. Thus, the application of LC-KSVD may be constrained in reality. To address this problem, we present S2KSVD through involving a computationally efficient label propagation (LP) process as a preprocessing step. The core idea is to employ the LP process to estimate the labels of unlabeled signals so that supervised prior knowledge that can significantly enhance classification can be increased. Simulation results on several machine fault datasets demonstrate that our algorithm delivers promising performance for machine fault classification.

This paper proposes an enhanced semi-supervised classification approach termed Nonnegative Sparse Neighborhood Propagation (SparseNP) that is an improvement to the existing neighborhood propagation due to the fact that the outputted soft labels of points cannot be ensured to be sufficiently sparse, discriminative, robust to noise and be probabilistic values. Note that the sparse property and strong discriminating ability of predicted labels is important, since ideally the soft label of each sample should have only one or few positive elements (that is, less unfavorable mixed signs are included) deciding its class assignment. To reduce the negative effects of unfavorable mixed signs on the learning performance, we regularize the l2,1-norm on the soft labels during optimization for enhancing the prediction results. The non-negativity and sum-to-one constraints are also included to ensure the outputted labels are probabilistic values. The proposed framework is solved in an alternative manner for delivering a more reliable solution so that the accuracy can be improved. Simulations show that satisfactory results can be obtained by the proposed SparseNP compared with other related approaches.

Traditional label propagation (LP) is shown to be effective for transductive classification. To enable the standard LP to handle outside images, two inductive methods by label reconstruction or by direct embedding have been presented, of which the latter scheme is relatively more efficient, especially for testing. But almost all inductive LP models use 1D vectors of images as inputs, which may destroy the topology structure of image pixels and usually suffer from high complexity due to the high dimension of 1D vectors in reality. To preserve the topology among pixels and address the scalability issue for the embedding based scheme, we propose a simple yet efficient Bilinear Embedding Label Propagation (BELP) by including a bilinear regularization term in terms of tensor representation to correlate the image labels with their bilinear features. BELP performs label prediction over the 2D matrices rather than 1D vectors, since images are essentially matrices. Finally, labels of new images can be easily obtained by embedding them onto a spanned bilinear subspace solved from a joint framework. Simulations verified the efficiency of our approach.

The insufficiency of labeled samples is a major problem in automatic image annotation. However, unlabeled samples are readily available and abundant. Hence, semi-supervised learning methods, which utilize partly labeled samples and a large amount of unlabeled samples, have attracted increased attention in the field of image classification. During the past decade, graph-based semi-supervised learning became one of the most important research areas in semi-supervised learning. In this letter, we propose a novel and effective graph based semi-supervised learning method for image classification. The new method is based on local and global regression regularization. The local regression regularization adopts a set of local classification functions to preserve both local discriminative and geometrical information; while the global regression regularization preserves the global discriminative information and calculates the projection matrix for out-of-sample extrapolation. Extensive simulations based on synthetic and real-world datasets verify the effectiveness of the proposed method.

In this article, we consider the problem of simultaneous low-rank recovery and sparse projection. More specifically, a new Robust Principal Component Analysis (RPCA)-based framework called Sparse Projection and Low-Rank Recovery (SPLRR) is proposed for handwriting representation and salient stroke feature extraction. In addition to achieving a low-rank component encoding principal features and identify errors or missing values from a given data matrix as RPCA, SPLRR also learns a similarity-preserving sparse projection for extracting salient stroke features and embedding new inputs for classification. These properties make SPLRR applicable for handwriting recognition and stroke correction and enable online computation. A cosine-similarity-style regularization term is incorporated into the SPLRR formulation for encoding the similarities of local handwriting features. The sparse projection and low-rank recovery are calculated from a convex minimization problem that can be efficiently solved in polynomial time. Besides, the supervised extension of SPLRR is also elaborated. The effectiveness of our SPLRR is examined by extensive handwritten digital repairing, stroke correction, and recognition based on benchmark problems. Compared with other related techniques, SPLRR delivers strong generalization capability and state-of-the-art performance for handwriting representation and recognition.

A new graph based constrained semi-supervised learning (G-CSSL) framework is proposed. Pairwise constraints (PC) are used to specify the types (intra- or inter-class) of points with labels. Since the number of labeled data is typically small in SSL setting, the core idea of this framework is to create and enrich the PC sets using the propagated soft labels from both labeled and unlabeled data by special label propagation (SLP), and hence obtaining more supervised information for delivering enhanced performance. We also propose a Two-stage Sparse Coding, termed TSC, for achieving adaptive neighborhood for SLP. The first stage aims at correcting the possible corruptions in data and training an informative dictionary, and the second stage focuses on sparse coding. To deliver enhanced inter-class separation and intra-class compactness, we also present a mixed soft-similarity measure to evaluate the similarity/dissimilarity of constrained pairs using the sparse codes and outputted probabilistic values by SLP. Simulations on the synthetic and real datasets demonstrated the validity of our algorithms for data representation and image recognition, compared with other related state-of-the-art graph based semi-supervised techniques.

An enhanced label propagation technique termed transformed neighborhood propagation (TNP) is proposed for semi-supervised learning. In the TNP setting, the processes of constructing weighted similarity graph and propagating label information of the labeled data to unlabeled points are conducted in the transformed feature space. TNP is mainly motivated by a fact that the optimal feature representation Y with possible unfavorable features and noises in the original data X removed by feature learning are more appropriate and accurate for measuring pair wise similarities of samples. To achieve the representation Y, The recent marginal semi-supervised sub-manifold projections is applied, so enhanced inter-class separation and enhanced intra-class compactness are delivered at the same time. The similarity graph is finally constructed based on Y. We also propose to calculate semi-supervised reconstruction weights for the weight assignment. As a result, the label estimation power can be enhanced by benefiting from the refined weighted similarity graph over Y instead of X, through propagating the labels of points in the transformed space for prediction. Visualization and image classification verified the effectiveness of our TNP, compared with other related label propagation algorithms.

For low-rank recovery and error correction, Low-Rank Representation (LRR) row-reconstructs given data matrix X by seeking a low-rank representation, while Inductive Robust Principal Component Analysis (IRPCA) aims to calculate a low-rank projection to column-reconstruct X. But either column or row information of Xis lost by LRR and IRPCA. In addition, the matrix X itself is chosen as the dictionary by LRR, but (grossly) corrupted entries may greatly depress its performance. To solve these issues, we propose a simultaneous low-rank representation and dictionary learning framework termed Tensor LRR (TLRR) for robust bilinear recovery. TLRR reconstructs given matrix X along both row and column directions by computing a pair of low-rank matrices alternately from a nuclear norm minimization problem for constructing a low-rank tensor subspace. As a result, TLRR in the optimizations can be regarded as enhanced IRPCA with noises removed by low-rank representation, and can also be considered as enhanced LRR with a clean informative dictionary using a low-rank projection. The comparison with other criteria shows that TLRR exhibits certain advantages, for instance strong generalization power and robustness enhancement to the missing values. Simulations verified the validity of TLRR for recovery.

This paper discusses a new setting of graph based semi-supervised learning (SSL) guided using pairwise constraints (PCs). Technically, we propose a novel Graph based Constrained Semi-Supervised Learning (G-CSSL) framework. In this setting, PCs are used to specify the types (intra- or inter-class) of points with labels. Because the number of labeled data is typically small in SSL setting, the core idea of this framework is to create and enrich the PCs sets using the propagated soft labels from both labeled and unlabeled data via special label propagation (SLP), and hence obtaining more supervised information for delivering enhanced learning performance. To obtain the predicted labels of unlabeled data, we calculate the sparse codes of all data vectors jointly to assign weights for SLP. To deliver enhanced inter-class separation and intra-class compactness, we also present a mixed soft-similarity measure to evaluate the similarity/dissimilarity of constrained sample pairs by using the sparse codes and outputted probabilistic values by SLP. Extensive simulations demonstrated the effectiveness of our G-CSSL for image representation and recognition, compared with other related SSL techniques.

Dementia is one of the most common neurological disorders among the elderly. Identifying those who are of high risk suffering dementia is important for early diagnosis in order to slow down the disease progression and help preserve some cognitive functions of the brain. To achieve accurate classification, significant amount of subject feature information are involved. Hence identification of demented subjects can be transformed into a pattern classification problem. In this letter, we introduce a graph based semi-supervised learning algorithm for Medical Diagnosis by using partly labeled samples and large amount of unlabeled samples. The new method is derived by a compact graph that can well grasp the manifold structure of medical data. Simulation results show that the proposed method can achieve better sensitivities and specificities compared with other state-of-art graph based semi-supervised learning methods.

This paper proposes a Robust Principal Component Analysis (RPCA) based framework called Sparse Projection and Low-Rank Recovery (SPLRR) for representing and recognizing handwritings. SPLRR calculates a similarity preserving sparse projection for salient feature extraction and processing new data for classification in addition to delivering a low-rank principal component and identifying errors or missing pixel values from a given data matrix. As a result, SPLRR will be applicable for handwritten recovery, recognition and the applications requiring online computation. To encode the similarity between features in the learning process, the Cosine similarity based regularization is incorporated to the SPLRR formulation. The sparse projection and the lowest-rank components are calculated from a scalable convex minimization problem that can be efficiently solved in polynomial time. The effectiveness of the proposed SPLRR is examined by handwritten digital repairing, stroke correction and recognition on two real problems. Results show that SPLRR can deliver state-of-the-art results in handwriting representation.

This paper discusses the image representation and decomposition problem using enhanced low-rank representation. Technically, we propose a Regularized Low-Rank Representation framework referred to as rLRR that is motivated by the fact that Latent Low-Rank Representation (LatLRR) delivers robust and promising results for image representation and feature extraction through recovering the hidden effects, but the locality among both similar principal and salient features to be encoded are not preserved in the original LatLRR formulation. To address this problem for obtaining enhanced performance, rLRR is proposed through incorporating an appropriate Laplacian regularization term that allows us to keep the local geometry of close features. Similar to LatLRR, rLRR decomposes a given data matrix from two directions by calculating a pair of low-rank matrices. But the similarities among principal features and salient features can be clearly preserved by rLRR. Thus the correlated features can be well grouped and the robustness of representations can also be effectively improved. The effectiveness of rLRR is examined by representation and recognition of real images. Results verified the validity of our presented rLRR technique.

This paper introduces an effective mechanism to extract informative principal features from the color images and proposes a color principal feature extraction technique referred to as ColorPCA. ColorPCA performs in the color image space, extracting the principal features directly from the color images. As a result, the color and local topological information of pixels at each level of the color images can be effectively preserved. In extracting the most representative features, a color image scatter matrix is constructed and its eigenvectors are employed for color principal feature extraction. ColorPCA has only one parameter (i.e., the reduced dimension) to estimate and the projection axes can be effectively obtained using eigen-decomposition. Extensive color image reconstruction and recognition over the benchmark problems verified the effectiveness of the presented ColorPCA. Results show that ColorPCA can effectively reconstruct the color images. Image recognition also demonstrates that ColorPCA can deliver promising results compared with other state-of-the-art 1D and 2D principal feature extraction algorithms.

Semi-supervised dimensionality reduction is an important research topic in many pattern recognition and machine learning applications. Among all the methods for semi-supervised dimensionality reduction, SDA and LapRLS are two popular ones. Though the two methods are actually the extensions of different supervised methods, we show in this paper that they can be unified into a regularized least square framework. However, the regularization term added to the framework focuses on smoothing only, it cannot fully utilize the underlying discriminative information which is vital for classification. In this paper, we propose a new effective semi-supervised dimensionality reduction method, called LLGDI, to solve the above problem. The proposed LLGDI method introduces a discriminative manifold regularization term by using the local discriminative information instead of only relying on neighborhood information. In this way, both the local geometrical and discriminative information of dataset can be preserved by the proposed LLGDI method. Theoretical analysis and extensive simulations show the effectiveness of our algorithm. The results in simulations demonstrate that our proposed algorithm can achieve great superiority compared with other existing methods.

Dealing with high-dimensional data has always been a major problem with the research of pattern recognition and machine learning. And Linear Discriminant Analysis (LDA) is one of the most popular methods for dimensionality reduction. But it only uses labeled samples while neglect the unlabeled samples, which are abundant and can be easily obtained in the real world. In this paper, we propose a new dimensionality reduction method by using the unlabeled samples to enhance the performance of LDA. The new method first propagates the label information from labeled set to unlabeled set via a label propagation process, where the predicted labels of unlabeled samples, called soft labels, can be obtained. It then incorporates the soft labels into the construction of scatter matrixes to find a transformed matrix for dimensionality reduction. In this way, the proposed method can preserve more discriminative information, which is good when solving the classification problem. Extensive simulations are carried based several datasets and the results show the effectiveness of the proposed method.

Dementia is one of the most common neurological disorders among the elderly. Identifying those who are of high risk suffering dementia is important to the administration of early treatment in order to slow down the progression of dementia symptoms. However, to achieve accurate classification, significant amount of subject feature information are involved. Hence identification of demented subjects can be transformed into a pattern recognition problem with high-dimensional nonlinear datasets. In this paper, we introduce trace ratio linear discriminant analysis (TR-LDA) for dementia diagnosis. An improved ITR algorithm (iITR) is developed to solve the TR-LDA problem. This novel method can be integrated with advanced missing value imputation method and utilized for the analysis of the nonlinear datasets in many real-world medical diagnosis problems. Finally, extensive simulations are conducted to show the effectiveness of the proposed method. The results demonstrate that our method can achieve higher accuracies for identifying the demented patients than other state-of-art algorithms.

Isomap is a well-known nonlinear dimensionality reduction (DR) method, aiming at preserving geodesic distances of all similarity pairs for delivering highly nonlinear manifolds. Isomap is efficient in visualizing synthetic data sets, but it usually delivers unsatisfactory results in benchmark cases. This paper incorporates the pairwise constraints into Isomap and proposes a marginal Isomap (M-Isomap) for manifold learning. The pairwise Cannot-Link and Must-Link constraints are used to specify the types of neighborhoods. M-Isomap computes the shortest path distances over constrained neighborhood graphs and guides the nonlinear DR through separating the interclass neighbors. As a result, large margins between both interand intraclass clusters are delivered and enhanced compactness of intracluster points is achieved at the same time. The validity of M-Isomap is examined by extensive simulations over synthetic, University of California, Irvine, and benchmark real Olivetti Research Library, YALE, and CMU Pose, Illumination, and Expression databases. The data visualization and clustering power of M-Isomap are compared with those of six related DR methods. The visualization results show that M-Isomap is able to deliver more separate clusters. Clustering evaluations also demonstrate that M-Isomap delivers comparable or even better results than some state-of-the-art DR algorithms.

Two novel unsupervised dimensionality reduction techniques, termed sparse distance preserving embedding (SDPE) and sparse proximity preserving embedding (SPPE), are proposed for feature extraction and classification. SDPE and SPPE perform in the clean data space recovered by sparse representation and enhanced Euclidean distances over noise removed data are employed to measure pairwise similarities of points. In extracting informative features, SDPE and SPPE aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics. This paper calculates the sparsest representation of all vectors jointly by a convex optimization. The sparsest codes enable certain local information of data to be preserved, and can endow SDPE and SPPE a natural discriminating power, adaptive neighborhood and robust characteristic against noise and errors in delivering low-dimensional embeddings. We also mathematically show SDPE and SPPE can be effectively extended for discriminant learning in a supervised manner. The validity of SDPE and SPPE is examined by extensive simulations. Comparison with other related state-of-the-art unsupervised algorithms show that promising results are delivered by our techniques.

Visualizing similarity data of different objects by exhibiting more separate organizations with local and multimodal characteristics preserved is important in multivariate data analysis. Laplacian Eigenmaps (LAE) and Locally Linear Embedding (LLE) aim at preserving the embeddings of all similarity pairs in the close vicinity of the reduced output space, but they are unable to identify and separate interclass neighbors. This paper considers the semi-supervised manifold learning problems. We apply the pairwise Cannot-Link and Must-Link constraints induced by the neighborhood graph to specify the types of neighboring pairs. More flexible regulation on supervised information is provided. Two novel multimodal nonlinear techniques, which we call trace ratio (TR) criterion-based semi-supervised LAE (S2LAE) and LLE (S2LLE), are then proposed for marginal manifold visualization. We also present the kernelized S2LAE and S2LLE. We verify the feasibility of S2LAE and S2LLE through extensive simulations over benchmark real-world MIT CBCL, CMU PIE, MNIST, and USPS data sets. Manifold visualizations show that S2LAE and S2LLE are able to deliver large margins between different clusters or classes with multimodal distributions preserved. Clustering evaluations show they can achieve comparable to or even better results than some widely used methods.

This paper incorporates the group sparse representation into the well-known canonical correlation analysis (CCA) framework and proposes a novel discriminant feature extraction technique named group sparse canonical correlation analysis (GSCCA). GSCCA uses two sets of variables and aims at preserving the group sparse (GS) characteristics of data within each set in addition to maximize the global interset covariance. With GS weights computed prior to feature extraction, the locality, sparsity and discriminant information of data can be adaptively determined. The GS weights are obtained from an NP-hard group-sparsity promoting problem that considers all highly correlated data within a group. By defining one of the two variable sets as the class label matrix, GSCCA is effectively extended to multiclass scenarios. Then GSCCA is theoretically formulated as a least-squares problem as CCA does. Comparative analysis between this work and the related studies demonstrate that our algorithm is more general exhibiting attractive properties. The projection matrix of GSCCA is analytically solved by applying eigen-decomposition and trace ratio (TR) optimization. Extensive benchmark simulations are conducted to examine GSCCA. Results show that our approach delivers promising results, compared with other related algorithms.

The cooperation behaviors existing in the animal and human being societies, have been modeled for the numerical optimization, but the local cooperation has not been modeled separately in optimization problems. In this paper the local cooperation is newly modeled as Neighborhood Field Model (NFM). Based on NFM, a new optimization technique called Neighborhood Field Optimization algorithm (NFO) is firstly proposed to deliver global optimization. In NFO, each individual is attracted by its superior neighbor and repulsed by its inferior neighbor to search a better solution. In this paper, NFO is compared with certain algorithms under twelve different benchmark functions. The results show that NFO can outperform them on multimodal functions in the respect of accuracy, effectiveness and robustness. It also can be noted that the cooperation behavior can play a dominant role in the optimization algorithm separately.

Semi-supervised dimensionality reduction is an important research topic in many pattern recognition and machine learning applications. Among all the methods for semi-supervised dimensionality reduction, SDA and Lap-LDA are two popular ones. Both SDA and Lap-LDA can perform dimensionality reduction by preserving the discriminative structure embedding in the labeled samples as well as the manifold structure embedded both in labeled and unlabeled samples. But they apply different schemes for semi-supervised dimensionality reduction. SDA has added the manifold term to the objective function of LDA while Lap-LDA has added such term to the objective function of Least Square with certain class indicator. In this paper, we further analyze the schemes of two methods and build the equivalence between them by giving a certain condition. We then show their difference when the certain condition cannot be satisfied. Extensive simulations have been conducted based several datasets. Both theoretical analysis and simulation results confirm the analysis. Finally, motivated by the equivalence and differences between two methods, we then propose an improved approach for semi-supervised dimensionality reduction. The proposed approach is actually a two-stage approach and can obtain the optimal solution equivalent to Lap-LDA (in this first stage) and SDA (in the second stage) with less computational cost.

This paper discusses the semi-supervised marginal projection problems learning from partial constrained data. Two effective multimodal dimensionality reduction (DR) algorithms, which we call semi-supervised marginal projections (SSMP) and orthogonal SSMP (OSSMP), are proposed. By specifying the types of similarity pairs with the pairwise constraints (PC), our techniques can preserve the global structures of all points as well as local geometrical and discriminant structures embedded in the PC. SSMP in singular case is also discussed. Because in all the PC guided methods, extracting the informative constraints is difficult and random constraints greatly affect the learning performance of techniques, this work also presents an effective and efficient methodology of optimally selecting the informative constraints for learning. The analytic form of the marginal projections can be effectively obtained by eigen-decomposition. The connections between this present work and the related semi-supervised algorithms are also detailed. The effectiveness of our proposed informative constraint selection method and algorithms are evaluated by benchmark problems. Results show our methods are capable of delivering competitive results with some widely used state-of-the-art semi-supervised algorithms.

Linear Discriminant Analysis (LDA) is a well-known dimensionality reduction algorithm for pattern recognition and machine learning. And Trace Ratio LDA (TR-LDA) and Null-space LDA (NLDA) are two popular variants of LDA. Both NLDA and TR-LDA can result in orthogonal transformations. However, they applied different schemes in deriving the optimal transformation. NLDA computes an orthogonal transformation in the null space of the within-class scatter matrix, while TRLDA computes an orthogonal transformation by an iterative procedure. In this paper, by using the trace difference problem as a bridge, we show that the above two algorithms can be equivalent when confronts with singularity problem. In addition, extensive simulations were conducted based on several datasets. Both theoretical analysis and simulation results confirm the equivalent relationship.

Dimensionality reduction has been a fundamental tool when dealing with high-dimensional dataset. And trace ration optimization has been widely used in dimensionality reduction because Trace ratio can directly reflect the similarity (Euclidean distance) of data points. Conventionally, there is no close-form solution to the original trace ratio problem. Prior works have indicated that trace ratio problem can be solved by an iterative way. In this paper, we propose an efficient algorithm to find the optimal solutions. The proposed algorithm can be easily extended to its corresponding kernel version for handling the nonlinear problems. Finally, we evaluate our proposed algorithm based on extensive simulations of real world datasets. The results show our proposed method is able to deliver marked improvements over other supervised and unsupervised algorithms.