Semi-supervised domain adaptation aims to transfer knowledge from a labeled source domain to a scarcely labeled target domain, despite distribution shifts. The challenge becomes greater when source and target data differ in acquisition modality, as in remote sensing where variations in sensor type (e.g., optical vs. radar), spectral properties (e.g., RGB vs. multispectral), or spatial resolution are common. This challenging scenario, known as Semi-Supervised Heterogeneous Domain Adaptation (SSHDA), requires learning across modalities with limited target labels. In this work, we propose SAHARA (Semi-supervised Adaptation in Heterogeneous domains via conditional Adversarial Representation disentanglement and Adaptive pseudo-labeling), a new method for SSHDA that combines conditional adversarial feature adaptation with dynamic pseudo-labeling to learn domain-invariant features and handle extremely scarce target annotations. Experiments on two heterogeneous remote sensing benchmarks for scene classification, conducted with both convolutional and transformer-based backbones, demonstrate that SAHARA consistently outperforms existing SSHDA and semi-supervised methods. The code is available at https: //TO-BE-DISCLOSED-UPON-ACCEPTANCE.

Hyperspectral pansharpening has received much attention in recent years due to technological and methodological advances that open the door to new application scenarios. However, research on this topic is only now gaining momentum. The most popular methods are still borrowed from the more mature field of multispectral pansharpening and often overlook the unique challenges posed by hyperspectral data fusion, such asi)the very large number of bands,ii)the overwhelming noise in selected spectral ranges,iii)the significant spectral mismatch between panchromatic and hyperspectral components,iv)a typically high resolution ratio. Imprecise data modeling especially affects spectral fidelity. Even state-of-the-art methods perform well in certain spectral ranges and much worse in others, failing to ensure consistent quality across all bands, with the risk of generating unreliable results. Here, we propose a hyperspectral pansharpening method that explicitly addresses this problem and ensures uniform spectral quality. To this end, a single lightweight neural network is used, with weights that adapt on the fly to each band. During fine-tuning, the spatial loss is turned on and off to ensure a fast convergence of the spectral loss to the desired level, according to a hysteresis-like dynamic. Furthermore, the spatial loss itself is appropriately redefined to account for nonlinear dependencies between panchromatic and spectral bands. Overall, the proposed method is fully unsupervised, with no prior training on external data, flexible, and low-complexity. Experiments on a recently published benchmarking toolbox show that it ensures excellent sharpening quality, competitive with the state-of-the-art, consistently across all bands. The software code and the full set of results are shared online on https://github.com/giu-guarino/rho-PNN.

In the last years it has been observed a growing interest toward deep leaning techniques for the pansharpening of multiresolution images. Due to the lack of data with ground truth, most deep learning solutions exploit synthetic reduced-resolution data to carry out supervised training. Such an approach, though granting an easy way to step over the lack of labeled data, has shown its limitations due to the statistical mismatch between real full-resolution data and synthetic reduced-resolution data, which eventually impacts on the generalization capacity of the trained models. This has motivated a recent paradigm shift from supervised to unsupervised learning frameworks for pansharpening. Unsupervised schemes, however, involve the definition of more sophisticated loss functions which comprise, at least, two fundamental terms: one responsible for the spectral quality, meant as consistency between the pansharpened image and the input multispectral component; the other accounting for the spatial quality, read as consistency between the output and the panchromatic input. Despite the very good results shown by many such unsupervised solutions, a minor attention has been devoted to the investigation of the interaction between the above mentioned loss terms and to their proper balance to grant stability while pursuing accuracy. This work aims to explore to what extent unsupervised spatial and spectral consistency losses can be reliably combined without impairing quality.

In this paper, a representative set of state-of-the-art methods for hyperspectral pansharpening, comprising both model- and deep learning-based ones, are reviewed and compared on four datasets from the PRISMA mission. The experimental analysis has been carried out using the most credited pansharpening quality indexes, complemented by a subjective visual inspection of sample results. The obtained outcomes have provided us a preview of the strengths and weaknesses of the latest solutions to the problem at hand, paving the way for future research lines from both the methodological and quality assessment perspectives.

This work proposes a hybrid approach to address the pansharpening of hyperspectral images, which mixes the use of a recently proposed CNN-based solution and a classical solution such as the Gram Schmidt Adaptive method (GSA). The hyperspectral datacube is split in two sets of bands, those falling in the visible range and the remaining ones. The first set is pansharpened using the GSA approach which has proven to grant very high quality results in this range. The remaining bands, whose relationship with the panchromatic band is much weaker, undergo a fusion process based on a recently proposed hyperspectral pansharpening method known as Rolling hyperspectral Pansharpening Neural Network (R-PNN). By doing so, we are able to take the best features from both solutions, getting higher quality results compared to the marginal use of any of the two methods.

Hyperspectral pansharpening is receiving a growing interest since the last few years as testified by a large number of research papers and challenges. It consists in a pixel-level fusion between a lower-resolution hyperspectral datacube and a higher-resolution single-band image, the panchromatic image, with the goal of providing a hyperspectral datacube at panchromatic resolution. Thanks to their powerful representational capabilities, deep learning models have succeeded to provide unprecedented results on many general purpose image processing tasks. However, when moving to domain specific problems, as in this case, the advantages with respect to traditional model-based approaches are much lesser clear-cut due to several contextual reasons. Scarcity of training data, lack of ground-truth, data shape variability, are some such factors that limit the generalization capacity of the state-of-the-art deep learning networks for hyperspectral pansharpening. To cope with these limitations, in this work we propose a new deep learning method which inherits a simple single-band unsupervised pansharpening model nested in a sequential band-wise adaptive scheme, where each band is pansharpened refining the model tuned on the preceding one. By doing so, a simple model is propagated along the wavelength dimension, adaptively and flexibly, with no need to have a fixed number of spectral bands, and, with no need to dispose of large, expensive and labeled training datasets. The proposed method achieves very good results on our datasets, outperforming both traditional and deep learning reference methods. The implementation of the proposed method can be found on https://github.com/giu-guarino/R-PNN.

The recent paradigm shift from model-based to data-driven approaches has involved a growing number of data-fusion tasks. Specifically for pansharpening, unsupervised deep learning methods have been recently explored with the goal of overcoming the generalization limits shown by early pansharpening convolutional neural networks based on supervised training schemes. Furthermore, some of these exploit the target-adaptive modality to face the scarcity of training data. On the downside, combining usupervised training and target adaptivity causes a non-negligible increase of the computational cost. This work presents a new target adaptive scheme that allows to keep limited the computational cost at inference time while preserving accuracy.

This work proposes a simple yet effective method to adapt unsupervised convolutional neural networks for pansharpening of multispectral images to the problem of hyperspectral image pansharpening, i.e., the fusion of a single high-resolution panchromatic band with a low-resolution hyperspectral data cube. This is achieved by means of a PCA transformation which allows to compact the most of the HS image energy in a few bands, which are then suitably super-resolved using a pansharpening network designed for few spectral bands. Our experiments show very encouraging results which compare favorably against the state-of-the-art methods.

Optical remote sensing images are subject to cloud phenomena that can cause information loss in Earth observation. The main alternative is represented by the synthetic aperture radar images. However, many Earth monitoring applications exploit specific spectral features defined for multispectral data only. In this work, we propose a method that aims to recover several spectral features through deep learning-based data fusion of Sentinel-1 and Sentinel-2 time-series. The proposed approach has been experimentally validated for radiometric indexes such as the normalized difference vegetation index, the normalized difference water index, the soil-adjusted vegetation index and the atmospherically resistant vegetation index. Both numerical and visual results show that the proposed solution outperforms consistently the compared methods.

This work proposes a simple yet effective method to adapt unsupervised convolutional neural networks from multispectral to hyperspectral pansharpening. Thus, it focuses on the fusion of a single high-resolution panchromatic band with a low-resolution hyperspectral data cube. This is achieved by means of a decorrelation transform, following the principal component analysis approach, which enables the compression of a significant portion of the hyperspectral image energy into a few bands. Afterwards, a suitably adapted pansharpening network designed for four spectral bands is used to super-resolve only the principal components. Experiments demonstrate high performance in both quantitative and qualitative evaluations, favorably comparing against state-of-the-art methods.

In latest years, deep learning has gained a leading role in the pansharpening of multiresolution images. Given the lack of ground truth data, most deep learning-based methods carry out supervised training in a reduced-resolution domain. However, models trained on downsized images tend to perform poorly on high-resolution target images. For this reason, several research groups are now turning to unsupervised training in the full-resolution domain, through the definition of appropriate loss functions and training paradigms. In this context, we have recently proposed a full-resolution training framework which can be applied to many existing architectures. Here, we propose a new deep learning-based pansharpening model that fully exploits the potential of this approach and provides cutting-edge performance. Besides architectural improvements with respect to previous work, such as the use of residual attention modules, the proposed model features a novel loss function that jointly promotes the spectral and spatial quality of the pansharpened data. In addition, thanks to a new fine-tuning strategy, it improves inference-time adaptation to target images. Experiments on a large variety of test images, performed in challenging scenarios, demonstrate that the proposed method compares favorably with the state of the art both in terms of numerical results and visual output. Code is available online at https://github.com/matciotola/Lambda-PNN.

In this work is presented a new adversarial training framework for deep learning neural networks for super-resolution of Sentinel 2 images, exploiting the data fusion techniques on 10 and 20 meters bands. The proposed scheme is fully convolutional and tries to answer the need for generalization in scale, producing realistic and detailed accurate images. Furthermore, the presence of a$\mathcal{L}_{1}$loss limits the instability of GAN training, limiting possible problems of spectral dis-tortion. In our preliminary experiments, the GAN training scheme has shown comparable results in comparison with the baseline approach.

In recent years, there has been a growing interest in deep learning-based pansharpening. Thus far, research has mainly focused on architectures. Nonetheless, model training is an equally important issue. A first problem is the absence of ground truths, unavoidable in pansharpening. This is often addressed by training networks in a reduced-resolution domain and using the original data as ground truth, relying on an implicit scale invariance assumption. However, on full-resolution images, results are often disappointing, suggesting such invariance not to hold. A further problem is the scarcity of training data, which causes a limited generalization ability and a poor performance on off-training-test images. In this article, we propose a full-resolution training framework for deep learning-based pansharpening. The framework is fully general and can be used for any deep learning-based pansharpening model. Training takes place in the high-resolution domain, relying only on the original data, thus avoiding any loss of information. To ensure spectral and spatial fidelity, a suitable two-component loss is defined. The spectral component enforces consistency between the pansharpened output and the low-resolution multispectral input. The spatial component, computed at high resolution, maximizes the local correlation between each pansharpened band and the panchromatic input. At testing time, the target-adaptive operating modality is adopted, achieving good generalization with a limited computational overhead. Experiments carried out on WorldView-3, WorldView-2, and GeoEye-1 images show that methods trained with the proposed framework guarantee a pretty good performance in terms of both full-resolution numerical indexes and visual quality.

This work presents a new unsupervised framework for training deep learning models for super-resolution of Sentinel-2 images by fusion of its 10-m and 20-m bands. The proposed scheme avoids the resolution downgrade process needed to generate training data in the supervised case. On the other hand, a proper loss that accounts for cycle-consistency between the network prediction and the input components to be fused is proposed. Despite its unsupervised nature, in our preliminary experiments the proposed scheme has shown promising results in comparison to the supervised approach. Besides, by construction of the proposed loss, the resulting trained network can be ascribed to the class of multi-resolution analysis methods.