Pansharpening tasks are the fusion of a low-resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image to generate a high-resolution multispectral (HRMS) image. Recently, the pansharpening method based on deep learning (DL) has received widespread attention because of its powerful fitting ability and efficient feature extraction. Since there is currently no method to make full use of different levels of feature information of PAN images to deeply fuse with MS images, we propose a new end-to-end deeply coupled feedback network to achieve high-quality image fusion at the feature level and this network named PSCF-Net. First, features are extracted from PAN images and MS images by different feature extraction blocks. Then, these features are deeply fused through two subnetworks composed of coupled feedback blocks, which can achieve high-quality fusion of features of different levels and images through coupling and feedback mechanisms. Finally, the feature maps of the two subnetworks are output as the final HRMS image through a channel integration layer. To make full use of the spatial information of PAN images and the spectral information of LRMS images, the extracted features include the features of MS images and the low- and high-level features of PAN images, and the low-level features of PAN images are injected with spectral information before being input to the subnetwork. At training time, we use SmoothL1 combined with structural similarity as the loss function in the network, and we experiment on the IKONOS and WorldView-2 datasets, respectively. The experimental results of reduced- and full-scale show that the deeply coupled feedback network we propose is superior to some of the current popular traditional methods and DL-based methods. Source code is available athttps://github.com/ahu-dsp/PSCF-Net.

In the multi-view multi-label (MVML) classification problem, multiple views are simultaneously associated with multiple semantic representations. Multi-view multi-label learning inevitably has the problems of consistency, diversity, and non-alignment among views and the correlation among labels. Most of the existing multi-view multi-label methods for non-aligned views assume that each view has a common or shared label set, but because a single view cannot contain the entire label information, they often learn suboptimal results. Based on this, this paper proposes a non-aligned multi-view multi-label classification method that learns view-specific labels (LVSL), aiming to explicitly mine the information of view-specific labels and low-rank label structures in non-aligned views in a unified model framework. Furthermore, to alleviate insufficient available label information, we thoroughly explored the global and local structural information among labels. Specifically, first, we assume that there is structural consistency between the view and the label space and then construct the view-specific label model in turn. Second, to enrich the original label space information, we mine the consistent information of multiple views and the low-rank correlation information hidden among multiple labels. Finally, the contribution weight of each view is combined with learning the complementary information among the views in the decision-making stage, and extend the model to handle nonlinear data. The results of the proposed method compared with existing state-of-the-art algorithms on several datasets validate its effectiveness.

Since capillary electrophoresis (CE) signals are always contaminated by random noise, which has negative influence on the accuracy of detection and analysis, it is necessary to remove noise before further applications of the CE signals. In this study, a tight frame learned from the data itself is applied to the removal of noise for CE signals. To achieve an effective decomposition of the CE signal, a one‐dimensional discrete tight frame tailored to the input signal is first constructed by introducing tight frame constraint into the popular dictionary learning model. Then, due to each subband containing different information of the noise, an adaptive threshold is computed to shrink the detail coefficients instead of using a global threshold. Finally, the denoised CE signal is reconstructed from the thresholded coefficients by using the inverse transform of the tight frame. To evaluate the denoising efficiency, the proposed method is applied to the simulated CE signals and real CE signals. Experimental results indicate that compared with other denoising methods, the proposed method obtains a better shape preservation of the peaks as well as a higher signal‐to‐noise ratio.

In this paper, a novel digital video resolution enhancement algorithm based on adaptive directional interpolation is proposed, where the directionality of the edge structure and the nonlocal self-similarity prior within the current frame as well as its adjacent frames are both considered. First, we establish the regularization equation that conforms to the prior model of a video frame and then take the classic bicubic interpolation result as the initial estimation to iteratively solve the restoration equation, in which the edge structures and contours in low resolution (LR) input are reconstructed to estimate and refine the desired high resolution (HR) output. Experimental results show that the proposed algorithm can effectively enhance the clarity of a video frame, with satisfying subjective visual quality and PSNR value.

In this paper, a novel system is developed for synthesizing user-specified emotions onto arbitrary input images or videos. Other than defining the visual affective model based on empirical knowledge, a data-driven learning framework is proposed to extract the emotion-related knowledge from a set of emotion-annotated images. In a divide-and-conquer manner, the images are clustered into several emotion-specific scene subgroups for model learning. The visual affection is modeled with Gaussian mixture models based on color features of local image patches. For the purpose of affective filtering, the feature distribution of the target is aligned to the statistical model constructed from the emotion-specific scene subgroup, through a piecewise linear transformation. The transformation is derived through a learning algorithm, which is developed with the incorporation of a regularization term enforcing spatial smoothness, edge preservation, and temporal smoothness for the derived image or video transformation. Optimization of the objective function is sought via standard nonlinear method. Intensive experimental results and user studies demonstrate that the proposed affective filtering framework can yield effective and natural effects for images and videos.

In this paper, we propose an efficient algorithm to reconstruct the 3D structure of a human face from one or more of its 2D images with different poses. In our algorithm, the nonlinear least-squares model is first employed to estimate the depth values of facial feature points and the pose of the 2D face image concerned by means of the similarity transform. Furthermore, different optimization schemes are presented with regard to the accuracy levels and the training time required. Our algorithm also embeds the symmetrical property of the human face into the optimization procedure, in order to alleviate the sensitivities arising from changes in pose. In addition, the regularization term, based on linear correlation, is added in the objective function to improve the estimation accuracy of the 3D structure. Further, a model-integration method is proposed to improve the depth-estimation accuracy when multiple nonfrontal-view face images are available. Experimental results on the 2D and 3D databases demonstrate the feasibility and efficiency of the proposed methods.

Eigengene extracted by independent component analysis (ICA) is one kind of effective feature for tumor classification. In this letter, a novel tumor classification approach is proposed by using eigengene and support vector machine (SVM) based classifier committee learning (CCL) algorithm. In this method, a strategy of random feature subspace division is designed to improve the diversity of weaker classifiers. Gene expression data constructed by different feature subspaces are modeled by ICA, respectively. And the corresponding eigengene sets extracted by the ICA algorithm are used as the inputs of the weaker SVM classifiers. Moreover, a strategy of Bayesian sum rule (BSR) is designed to integrate the outputs of the weaker SVM classifiers, and used to provide a final decision for the tumor category. Experimental results on three DNA microarray datasets demonstrate that the proposed method is effective and feasible for tumor classification.