Model-based shallow machine-learning methods and data-driven deep-learning (DL) methods have been advanced to address hyperspectral and multispectral image fusion (HS–MS fusion). Nonetheless, model-based approaches, which meticulously craft regularization terms within optimization models using hand-engineered priors, often struggle to pinpoint the optimal solution efficiently. DL-based methods, which train on extensive datasets to learn a non-linear mapping for generating a high spatial and spectral resolution image (HS2I), exhibit limited generalization capabilities when applied to novel and diverse test datasets. To improve the generalization ability and optimization efficiency of the existing HS–MS fusion methods, a novel deep low-rank prior (DLRP)-based self-supervised HS–MS fusion approach is devised. It incorporates a low-rank learning paradigm to produce the fused HS2I, subject to the constraints imposed by loss functions. Instead of deriving the solution via solving a low-rank approximation optimization problem, deep image prior (DIP) learned by a two-dimensional CNN and a one-dimensional CNN are integrated as the low-rank prior.

Radio maps enrich radio propagation and spectrum occupancy information, which provides fundamental support for the operation and optimization of wireless communication systems. Traditional radio maps are mainly achieved by extensive manual channel measurements, which is time-consuming and inefficient. To reduce the complexity of channel measurements, radio map estimation (RME) through novel artificial intelligence techniques has emerged to attain higher resolution radio maps from sparse measurements or few observations. However, black box problems and strong dependency on training data make learning-based methods less explainable, while model-based methods offer strong theoretical grounding but perform inferior to the learning-based methods. In this paper, we develop a deep unrolled low-rank tensor completion network (DULRTC-RME) for radio map estimation, which integrates theoretical interpretability and learning ability by unrolling the tedious low-rank tensor completion optimization into a deep network. It is the first time that algorithm unrolling technology has been used in the RME field. Experimental results demonstrate that DULRTC-RME outperforms existing RME methods.

The increasing deployment of unmanned aerial vehicles (UAVs) as mobile communication relays in urban environments necessitates accurate 3-D modeling of complex urban areas for optimal communication. Current practices involve LiDAR-based 3-D scanning to generate point cloud data; however, sensor limitations and adverse weather conditions may compromise data quality. This study proposes a new multimodal point cloud and image fusion completion network (PIFC-Net) based on a generative adversarial network (GAN), specifically tailored for large-scale urban environments. The experimental study tested five different building shapes and various objects, and the results commend the network for its effectiveness in enhancing the quality and efficiency of point cloud completion.

Fusion with corresponding finer-resolution images has been a promising way to enhance hyperspectral images (HSIs) spatially. Recently, low-rank tensor-based methods have shown advantages compared with other kind of ones. However, these current methods either relent to blind manual selection of latent tensor rank, whereas the prior knowledge about tensor rank is surprisingly limited, or resort to regularization to make the role of low rankness without exploration on the underlying low-dimensional factors, both of which are leaving the computational burden of parameter tuning. To address that, a novel Bayesian sparse learning-based tensor ring (TR) fusion model is proposed, named as FuBay. Through specifying hierarchical sprasity-inducing prior distribution, the proposed method becomes the first fully Bayesian probabilistic tensor framework for hyperspectral fusion. With the relationship between component sparseness and the corresponding hyperprior parameter being well studied, a component pruning part is established to asymptotically approaching true latent rank. Furthermore, a variational inference (VI)-based algorithm is derived to learn the posterior of TR factors, circumventing nonconvex optimization that bothers the most tensor decomposition-based fusion methods. As a Bayesian learning methods, our model is characterized to be parameter tuning-free. Finally, extensive experiments demonstrate its superior performance when compared with state-of-the-art methods.

Recently, convolutional neural networks (CNNs) have been developed for remote sensing image fusion (RSIF). To obtain competitive fusion performance, network design becomes more complicated by stacking convolutional layers deeper and wider. However, problems still remain when applying existing networks in practical applications. On the one hand, researchers focus on improving spatial resolution but ignore that the fused images will be used in subsequent interpretation applications, e.g., objection detection. On the other hand, RSIF involves different tasks with different image sources e.g., pansharpening of the panchromatic and multispectral image, hypersharpening of the panchromatic and hyperspectral image, etc. However, existing networks only solve one of them, failing to be compatible with other tasks. To address the above problems, a convenient task-inspired multiscale nonlocal-attention network (MNAN) is proposed for RSIF. The proposed MNAN focuses more on enhancing the multi-scale targets in the scene when improving the resolution of the fused image. In addition, the proposed network can be applied to both pansharpening and hypersharpening tasks without any modification.

Although spatio-spectral and spatio-temporal fusion has been well explored, few efforts are made on integrating spatio-spectral-temporal features. As an intrinsic prior, low tensor-rank has been successfully taken into effect by current fusion models, most of which, however, resort to establishing an overall low-rank norm without performing factorization techniques thus have trouble capturing the latent high-order structure of hyperspectral data cube. To address that, a novel Anisotropicly Sparse (AS) tensor norm is developed to make the rank minimization a learnable process under Tucker decomposition. The AS norm enables the model to minimize the multi-linear tensor ranks if imposed on the core tensor after factorization, hence it significantly improves the model’s fusion performance. In the temporal domain, a Hadamard-product based variability descriptor is incorporated into the fusion model to map the former information to current time. Additionally, piece-wise smooth prior of the Tucker factors is employed by extra regularizers as supplement to the loss spatial information. With the Proximal Differential Matrix being developed for optimization, the proposed method reaches state-of-the-art results on both spatio-spectral and spatio-spectral-temporal fusion at low computational cost.

The injection scheme provides an efficient way for CS- and MRA-based pansharpening approaches. Within this paradigm, the estimation of injection gains is one of the keys to pansharpening outcomes, which has attracted much attention in the community. Most of the existing models are derived from the regression methodology. Hence, the reference is indispensable for the estimation. However, the reference is unavailable in practice, and therefore, the estimation is usually performed at a degraded scale. This article is devoted to the estimation of injection gains without reference. A hybrid-scale (HS) estimation, which involves both the high-resolution and low-resolution data, is proposed, along with three HS models. The proposed method features a context-based and fast implementation with fewer tunable parameters. Experimental results show that the HS models yield more accurate and robust results compared with the typical regression-based models, and they are also competitive with the state-of-the-art approaches.

Recently, deep learning has become one of the most popular tools for pansharpening, many relevant methods have been investigated and reflected great performance. However, a non-negligible problem is the absence of ground-truth (GT). A common solution is using degraded images as training input and the original images are employed as GT. The learned mapping between low resolution (LR) and high resolution (HR) is simulated, is not real, which may cause spectral distortion or insufficient spatial texture enhancement of fused images. In order to address the drawback, a novel unsupervised attention pansharpening net (UAP-Net) is proposed. The proposed UAP-Net mainly contains two major components: 1) the deep residual network (DRN) and 2) spatial texture attention block (STAB). The DRN aims to extract spectral features and spatial details features from low-resolution multi-spectral (LRMS) and panchromatic (PAN), and to fuse those features to make them more representative. The designed STAB adopts the high-frequency component of corresponding input PAN as the weight to enhance the spatial details of the residual block output features. Moreover, a new loss function including two spatial losses and two spectral losses are established. The losses are calculated in the spatial domain and the frequency domain, respectively. Experiments on Gaofen-2 and Worldview-2 remote sensing data demonstrate that the proposed UAP-Net could fuse PAN and LRMS images effectively without the help of high-resolution multi-spectral (HRMS). The proposed framework is fully general and can be used for many multisource remote sensing image fusion, and achieves optimal performance in terms of both the subjective visual effect and the quantitative evaluation.

Improving the spatial resolution of a hyperspectral image (HSI) is of great significance in the remotely sensed field. By fusing a high-spatial-resolution multispectral image (MSI) with an HSI collected from the same scene, hyperspectral and multispectral (HS–MS) fusion has been an emerging technique to address the issue. Extracting complex spatial information from MSIs while maintaining abundant spectral information of HSIs is essential to generate the fused high-spatial-resolution HSI (HS2I). A common way is to learn low-rank/sparse representations from HSI and MSI, then reconstruct the fused HS2I based on tensor/matrix decomposition or unmixing paradigms, which ignore the intrinsic geometry proximity inherited by the low-rank property of the fused HS2I. This study proposes to estimate the high-resolution HS2I via low-rank tensor approximation with geometry proximity as side information learned from MSI and HSI by defined graph signals, which we name GLRTA. Row graph ${\mathcal{G}_r}$ and column graph ${\mathcal{G}_c}$ are defined on the horizontal slice and lateral slice of MSI tensor $\mathcal{M}$ respectively, while spectral band graph ${\mathcal{G}_b}$ is defined on a frontal slice of HSI tensor $\mathcal{H}$. Experimental results demonstrate that the proposed GLRTA can effectively improve the reconstruction results compared to other competitive works.

Spectral quality is one of the most critical issues that has to be considered in real hyperspectral image (HSI) application. Denoising, destriping, inpainting, deblurring and super-resolution are common techniques to improve the quality of HSIs from different aspects. These techniques have attracted much attention that a diversity of methods, algorithms, tools have been well developed to facilitate the development of HSI restoration. Although effectively improving the quality of HSIs, these technologies mainly focus on recovering an HSI captured in the normal sunlight. It is acknowledged that HSIs are captured via passive imaging mechanisms covering the spectral bands from visible& near-infrared to shortwave infrared spectral range (i.e., around 400nm to 2500nm). The imaging condition limits HSI spectrometers to capture HSIs without sunlight (e.g., in dark environments or night time). In this work, a low-light HSI restoration method is proposed, where we borrow idea of intrinsic decomposition based on Retinex theory in natural image low-light enhancement. Additionally, camera response function that describe the spectral degradation of RGB image and relationship between irradiance and pixel values are employed, respectively. The experimental results validate the effectiveness of the proposed method.

Low-rank-tensor-approximation (LRTA)-based hyperspectral image (HSI) restoration has drawn increasing attention. However, most of the methods construct a hidden low-rank tensor by utilizing the non-local self-similarity (NLSS) and global spectral correlation (GSC) inherited by HSIs. Although achieving state-of-the-art (SOTA) restoration performance, NLSS and GSC have limitations. NLSS is introduced from natural image denoising to remove spatially independent identically distributed (i.i.d.) Gaussian and impulse noise. While GSC, which is naturally possessed by HSIs, is adopted to maintain the spectral integrity and remove spectrally, i.i.d., degradations. Therefore, NLSS and GSC may not be successfully used for complex HSI restoration tasks, such as destriping, cloud removal and recovery of atmospheric absorption bands. To solve the issue, borrowing the idea from manifold learning, the geometry information characterized by proximity relationship, is integrated with the LRTA to solve the above issue, named as multi-graph-based LRTA (MGLRTA). Different with most of the existing methods, the proposed MGLRTA directly models an HSI as a low-rank tensor and efficiently explores the extra proximity information on the defined graphs that are not only inherited by the low-rank constraints but also naturally possessed in HSIs. A well-posed iterative algorithm is designed to solve the restoration problem. Experimental results on different datasets that cover several severe degradation scenarios demonstrate that the proposed MGLRTA outperforms the SOTA HSI restoration methods.

Random Gaussian noise and striping artifacts are common phenomena in hyperspectral images (HSI). In this article, an effective restoration method is proposed to simultaneously remove Gaussian noise and stripes by merging a denoising and a destriping submodel. A denoising submodel performs a multiband denoising, i.e., Gaussian noise removal, considering Gaussian noise variations between different bands, to restore the striped HSI from the corrupted image, in which the striped HSI is constrained by a weighted nuclear norm. For the destriping submodel, we propose an adaptive anisotropy total variation method to adaptively smoothen the striped HSI, and we apply, for the first time, the truncated nuclear norm to constrain the rank of the stripes to 1. After merging the above two submodels, an ultimate image restoration model is obtained for both denoising and destriping. To solve the obtained optimization problem, the alternating direction method of multipliers (ADMM) is carefully schemed to perform an alternative and mutually constrained execution of denoising and destriping. Experiments on both synthetic and real data demonstrate the effectiveness and superiority of the proposed approach.

Tensor-based fusion that couples the high spatial resolution of a multispectral image (MSI) to the high spectral resolution of a hyperspectral image (HSI) is considered. The fusion problem is first formulated mathematically as a convex optimization of a tensor trace norm imposing low-rank spatially as well as spectrally, with an alternating-directions optimization featuring linearization providing the solution. Although prior tensor-based fusion approaches typically resort to tensor decomposition, the proposed algorithm exploits ideas from the field of tensor completion to directly impose a low-rank property spatially and spectrally while avoiding the computationally complex patch clustering and dictionary learning common to competing fusion techniques. Additionally, small modifications to the basic optimization permit a fusion process robust to missing hyperspectral values such as those that can result from dead stripes in real hyperspectral sensors. The experimental evaluations on both synthetic imagery as well as real imagery demonstrate that the resulting low-rank tensor-approximation (LRTA) fusion algorithm preserves both spatial details and texture, yielding significantly improved image quality when compared to other state-of-the-art fusion methods as well as effective restoration under conditions of missing stripes within the HSI.

Recently, collaborative representation (CR) has drawn increasing attention in hyperspectral image classification due to its simplicity and effectiveness. However, existing representation-based classifiers do not explicitly utilize class label information of training samples in estimating representation coefficients. To solve this issue, a structure-aware CR with Tikhonov regularization (SaCRT) method is proposed to consider both class label information of training samples and spectral signatures of testing pixels to estimate more discriminative representation coefficients. In the proposed framework, marginal regression is employed; furthermore, an interclass row-sparsity structure is designed to preserve the compact relationship among intraclass pixels and more separable interclass pixels, thereby enhancing class separability. The experimental results evaluated using three hyperspectral data sets demonstrate that the proposed method significantly outperforms some state-of-the-art classifiers.

Pushbroom acquisition of hyperspectral imagery is prone to striping artifacts in the along-track direction. A hyperspectral destriping algorithm is proposed such that the subbands of a 3-D wavelet transform most affected by pushbroom stripes-namely, those with spatially vertical orientation-are the exclusive focus of destriping. The proposed method features an iterative image decomposition composed of a low-rank model for the stripes coupled with a group-sparse prior on the wavelet coefficients of the subbands in question. While low-rank stripe models have been widely used in the past, they typically have been deployed in conjunction with a total-variation prior on the image that is prone to oversmoothing and residual stripe artifacts. On the other hand, the proposed group-sparse prior not only captures the well-known sparse nature of wavelet coefficients but also capitalizes on their vertical clustering in the subbands in question. In addition, while many prior destriping methods are wavelet-based, they employ 2-D transforms band by band. In contrast, the proposed 3-D wavelet transform provides a greater concentration of stripe information into fewer wavelet coefficients, leading to more effective destriping. Experimental results on both synthetically striped imagery as well as real striped imagery from an actual hyperspectral sensor demonstrate superior image quality for the proposed method as compared with other state-of-the-art methods.

Visible light communication (VLC) is considered as a promising candidate to improve the performance of indoor communication as the complement of wireless radio frequency (RF) communications due to the scarcity of RF resources. Combining the VLC with software-defined small-cell networks will substantially improve the user data rates in indoor heterogeneous networks. In this paper, we introduce the software-defined philosophy into orthogonal frequency-division multiple access-based heterogeneous software-defined and twinned VLC and RF small-cell networks. The pivotal issues of energy efficient (EE) subchannel and power allocation are investigated in the context of software-defined VLC and RF small-cell networks. We formulate the EE resource allocation problem as a non-convex optimization problem, and then, transform it into a convex one using Dinkelbach's method. In addition, distributed subchannel and power allocation algorithms for both VLC and RF are proposed for solving the problem based on the powerful alternative direction method of multipliers. Simulation results verify the effectiveness of resource allocation algorithms conceived for the heterogeneous software-defined twinned VLC and RF small-cell networks in terms of its good convergence and overall performance.

The next generation of space-borne synthetic aperture radar (SAR) systems will emphasize on high-resolution and wide-swath imaging. For these design purposes, multichannel technology in azimuth is a promising candidate. In the multichannel SAR system, the nonuniform azimuth signal needs to be reconstructed when the pulse repetition frequency (PRF) deviates from a specific one. The traditional digital beamforming (DBF) algorithm works well just in the cases of uniform sampling and moderately nonuniform sampling. If there is highly nonuniform sampling or coinciding sampling, this method may fail. In this paper, a new method to handle the cases of highly nonuniform sampling is proposed. The proposed method is based on an equivalent sampling strategy and the minimum mean square error (MMSE) criterion. It is robust over a wide range of PRF. Compared to the conventional algorithms, this method dramatically improves the SNR, which is validated by simulation experiments.