The rise of HDR-WCG display devices has highlighted the need to convert SDRTV to HDRTV, as most video sources are still in SDR. Existing methods primarily focus on designing neural networks to learn a single-style mapping from SDRTV to HDRTV. However, the limited information in SDRTV and the diversity of styles in real-world conversions render this process an ill-posed problem, thereby constraining the performance and generalization of these methods. Inspired by generative approaches, we propose a novel method for SDRTV to HDRTV conversion guided by real HDRTV priors. Despite the limited information in SDRTV, introducing real HDRTV as reference priors significantly constrains the solution space of the originally high-dimensional ill-posed problem. This shift transforms the task from solving an unreferenced prediction problem to making a referenced selection, thereby markedly enhancing the accuracy and reliability of the conversion process. Specifically, our approach comprises two stages: the first stage employs a Vector Quantized Generative Adversarial Network to capture HDRTV priors, while the second stage matches these priors to the input SDRTV content to recover realistic HDRTV outputs. We evaluate our method on public datasets, demonstrating its effectiveness with significant improvements in both objective and subjective metrics across real and synthetic datasets.

Face restoration is a challenging task due to the need to remove artifacts and restore details. Traditional methods usually use generative model prior to achieve face restoration, but the restored results are still insufficient in terms of realism and details. In this paper, we introduce OmniFace, a novel face restoration framework that leverages Transformer-based diffusion flow. By exploiting the scaling property of Transformer, OmniFace achieves high-resolution restoration with exceptional realism and detail. The framework integrates three key components: (1) a Transformer-driven vector estimation network, (2) a representation aligned ControlNet, and (3) an adaptive training strategy for face restoration. The inherent scaling law of Transformer architectures enables the restoration of high-quality faces at high resolution. The controlnet combined with pre-trained diffusion representation can be easily trained. The adaptive training strategy provides a vector field that is more suitable for face restoration. Comprehensive experiments demonstrate that OmniFace outperforms existing techniques in terms of restoration quality across multiple benchmark datasets, especially in restoring photographic-level texture details in high-resolution scenes.

The challenges in semantic segmentation of remote sensing images arise primarily from significant variations in target sizes, diverse shapes, and complex background environments. To address these challenges, we have developed a feature-guided semantic segmentation model for remote sensing images, further enhancing its accuracy with high-fidelity masks generated by the SAM vision large model. The principal contributions of this study are outlined as follows: We introduce a semantic segmentation model that employs feature guidance, ingeniously integrating CNN and Transformer architectures to effectively capture both the local details and the global context of remote sensing images. We propose an improved method based on the SAM vision large model, featuring a dual-branch structure that incorporates two key modules: semantic voting and mask refinement. The semantic voting module corrects semantic errors in both branches. Subsequently, a mask refinement module is utilized to further enhance these masks. In experimental validations using the iSAID dataset, the model significantly improved the mean Intersection over Union (mIoU) to 61.54% and the meanF1 score to 74.69%, underscoring the significance and applicability of the proposed methodology.

The fusion of hyperspectral (HS) and panchromatic (PAN) images aims to generate a fused HS image that combines spectral information of the HS image with spatial information of the PAN image. In this article, we propose a multiresolution spatial–spectral feature learning (MSSL) framework for fusing HS and PAN images. The proposed MSSL transforms the existing deep and complex network into several simple and shallow subnetworks to simplify the feature learning process. MSSL upsamples the HS image while downsamples the PAN image and designs multiresolution 3-D convolutional autoencoder (CAEs) networks with a spectral constraint to learn complete spatial–spectral features of the HS image. MSSL designs multiresolution 2-D CAEs with spatial constraint to extract spatial features of the PAN image, with a low computational cost. In order to effectively generate the pansharpened HS image with high spatial and spectral fidelity, a multiresolution residual network is presented to reconstruct the HS image from the extracted spatial–spectral features. Extensive experiments are conducted on three widely used remote sensing data sets in comparison with state-of-the-art HS image fusion methods, demonstrating the superiority of the proposed MSSL method. Code is available athttps://github.com/Jiahuiqu/MSSL.

Recently developed deep learning-based hyperspectral anomaly detection (HAD) methods typically include two steps where the deep feature extraction is not designed specifically for the HAD task. In this article, we propose a spectral distribution-aware estimation network (SDEN) that does not conduct feature extraction and anomaly detection in two separate steps but instead learns both jointly to estimate anomalies directly in an end-to-end manner without postprocessing. The unified framework can ensure that the extracted features serve better for anomaly detection. To preserve the distribution of hyperspectral images (HSIs) during dimensionality reduction, the SDEN introduces a spectral distribution (SD)-aware module imposed with a local-invariant constraint. More specifically, we adopt Markov chain Monte Carlo (MCMC) that enables the SD module to better estimate the distribution of the complex HSIs. Considering the powerful representation capability of Gaussian mixture model (GMM), the SDEN leverages it to establish an estimation module in the deep latent space where the anomaly resides in low density while the background not. We demonstrate that the SDEN yields competitive and highly promising results in comparison with the anomaly detection benchmarks.

Despite recent advances achieved by deep learning techniques in the fusion of low-spatial-resolution hyperspectral image (LR-HSI) and high-spatial-resolution multispectral image (HR-MSI), it remains a challenge to reconstruct the high-spatial-resolution HSI (HR-HSI) with more accurate spatial details and less spectral distortions, since the low-level structure information such as sharp edges tends to be weakened or lost as the network depth grows. To tackle this issue, we creatively propose an edge-conditioned feature transform network (EC-FTN) in this article, which is mainly composed of three parts, namely, feature extraction network (FEN), feature fusion and transformation network (FFTN), and image reconstruction network (IRN). First, two computationally efficient FENs with 3-D convolutions and reshaping layers are employed to extract the joint spectral-spatial features of input images. Then, the FFTN conditioned on the edge map prior can fuse and transform the features adaptively, in which a fusion node and several cascaded feature modulation modules (FMMs) equipped with feature-wise modulation layers are constructed. Specifically, the edge map is generated via transfer learning, i.e., by applying the Sobel operator to feature maps of the red-green-blue (RGB) version of HR-MSI resulting from the pretrained VGG16 model without extra training. Finally, the desired HR-HSI is recovered from the transformed features through IRN. Furthermore, we elaborately design a weighted combinatorial loss function consisting of mean absolute error, image gradient difference, and spectral angle terms to guide the training. Experiments on both ground-based and remotely sensed datasets demonstrate that our EC-FTN outperforms state-of-the-art methods in visual and quantitive evaluations, as well as in fine details reconstruction.

Convolutional neural networks (CNNs) have recently achieved impressive improvements on hyperspectral (HS) pansharpening. However, most of the CNN-based HS pansharpening approaches would have to first upsample the low-resolution hyperspectral image (LR-HSI) using bicubic interpolation or data-driven training strategy, which inevitably lose some details or greatly rely on the learning process. In addition, most previous methods regard the pansharpening as a black-box problem and treat diverse features equally, thus hindering the discriminative ability of CNNs. To conquer these issues, a novel HS pansharpening method using deep hyperspectral prior (DHP) and dual-attention residual network (DARN) is proposed in this article. Specifically, we first upsample the LR-HSI to the scale of the panchromatic (PAN) image through the DHP algorithm, which can better preserve spatial and spectral information without learning from large data sets. The upsampled result is then concatenated with the PAN image to form the input of the DARN, where several channel-spatial attention residual blocks (CSA ResBlocks) are stacked to map the residual HSI between the reference HSI and the upsampled HSI. In each CSA ResBlock, two complementary attention modules, i.e., channel attention and spatial attention modules, are designed to adaptively learn more informative features of spectral channels and spatial locations simultaneously, which can effectively boost the fusion accuracy. Finally, the fused HSI is obtained by the summation of the upsampled HSI and the reconstructed residual HSI. The experimental results of both simulated and real HS data sets demonstrate that the performance of our DHP-DARN method is superior over the state-of-the-art HS pansharpening approaches.

Spectral unmixing amounts to automatically finding the signatures of pure spectral components (called endmembers in the hyperspectral imaging literature) and their associated abundance fractions in each pixel of the hyperspectral image. Many algorithms have been proposed to automatically find spectral endmembers in hyperspectral data sets. Perhaps one of the most popular ones is the pixel purity index (PPI), which is available in the ENVI software from Exelis Visual Information Solutions. This algorithm identifies the endmembers as the pixels with maxima projection values after projections onto a large randomly generated set of random vectors (called skewers). Although the algorithm has been widely used in the spectral unmixing community, it is highly time consuming as its precision asymptotically increases. Due to its high computational complexity, the PPI algorithm has been recently implemented in several high-performance computing architectures, including commodity clusters, heterogeneous and distributed systems, field programmable gate arrays, and graphics processing units (GPUs). In this letter, we present an improved GPU implementation of the PPI algorithm, which provides real-time performance for the first time in the literature.

Nakanishi and Niino proposed an improved Mellor-Yamada (M-Y) Level-3 model (MYNN) surface for three-dimensional simulation of advection fog. The model is on the basis of large-eddy simulation and is numerically stable. The model predicts vertical profiles of mean quantities such as temperature that are in good agreement with those obtained from large-eddy simulation of a radiation fog. In this paper, we accelerate Nakanishi and Niino PBL's Surface Layer scheme in a highly parallel environment, using NVIDIA Graphics Processing Units (GPU). This GPU implementation efficiently utilizes the fine grained parallelism exhibited by the MYNN PBL scheme. The algorithm is accelerated on a low-cost personal supercomputer with over 500 CUDA cores running on a GPU. This implementation achieves a high speedup of 160×.