Recently, improving the residual structure and designing efficient convolutions have become important branches of lightweight visual reconstruction model design. We have observed that the feature addition mode (FAM) in existing residual structure tends to lead to slow feature learning or stagnation in feature evolution, a phenomenon we define as network inertia. In addition, although blueprint separable convolutions (BSConv) have proved the dominance of intra-kernel correlation, BSConv forces the blueprint to perform scale transformation on all channels, which may lead to incorrect intra-kernel correlation and introduce useless or disruptive features on some channels and hinder the effective propagation of features. Therefore, in this paper, we rethink the FAM and BSConv for super-light visual reconstruction framework design. First, we design a novel linking mode, called feature diversity evolution link (FDEL), which aims to alleviate the phenomenon of network inertia by reducing the retention of previous low-level features, thereby promoting the evolution of feature diversity. Second, we propose blueprint controllable convolutions (B2Conv). The B2Conv can adaptively pick accurate intra-kernel correlation in the depth-axis, effectively preventing the introduction of useless or disruptive features. Based on FDEL and B2Conv, we develop a super-light super-resolution (SR) framework SLVR for visual reconstruction. Both FDEL and B2Conv can serve as efficient plugins. Extensive experimental results demonstrate the effectiveness of our proposed B2Conv, FDEL, and SLVR. Our code will be available at https://github.com/chongningni/SLVR.

Recently, due to higher requirements for satellite video resolution, video super-resolution (VSR) has been extensively studied. However, the following problems have not been effectively resolved: 1) Previous satellite VSR methods cannot achieve continuous-scale (integer and non-integer scale) VSR with a single model. 2) Satellite video has complex ground and weak textures, which increases the difficulty of capturing motion information. In addition, existing methods adopt a unified alignment path, which leads to a drop in feature alignment accuracy. 3) During feature fusion, previous methods ignore the correlation of spatio-temporal information in satellite video and cannot make full use of the spatio-temporal information. To address the above problems, in this paper, we propose a novel network for continuous-scale satellite VSR (CSVSR). Specifically, first, for effective motion capture and accurate feature alignment, we design a residual-guided and time-aware dynamic routing alignment module, which can use feature residuals to lock motion areas and then dynamically select the corresponding alignment path based on the temporal distance. Second, we proposed a non-local mask-based feature fusion module to exploit the correlation of the spatio-temporal features and complete effective spatio-temporal feature fusion. Third, to make our network adapt to multi-task learning, we develop a scale-aware convolutional (SA-Conv) layer, which lets our network dynamically extract scale-adaptive features according to the input scale factors. Finally, we propose a continuous-scale upsampling module with a global feature implicit function (GFIF), which can achieve continuous-scale mapping from features to pixel values. In addition, we carefully design a novel training strategy to optimize our network. Comprehensive experiments verify that the proposed CSVSR has superior reconstruction performance on continfuous-scale factors. The code will be available at https://github.com/chongningni/CSVSR.

Remote sensing images (RSIs) in real scenes may be disturbed by multiple factors, such as optical blur, undersampling, and additional noise, resulting in complex and diverse degradation models. At present, mainstream super-resolution (SR) algorithms only consider a single and fixed degradation (such as bicubic interpolation) and cannot flexibly handle complex degradations in real scenes. Therefore, designing an SR model that can deal with various degradations has gradually attracted researchers’ attention. Some early studies estimate degradation kernels and then perform degradation-adaptive SR but face the problems of estimation error amplification and insufficient high-frequency details in the results. Although blind SR algorithms based on generative adversarial networks (GANs) have greatly improved visual quality, they still suffer from pseudo-texture, mode collapse, and poor training stability. This article proposes a novel blind SR framework based on the stochastic normalizing flow (BlindSRSNF) to address the above problems. BlindSRSNF learns the conditional probability distribution over the high-resolution image space given a low-resolution (LR) image by explicitly optimizing the variational bound on the likelihood. BlindSRSNF is easy to train and can generate photorealistic SR results that outperform GAN-based models. In addition, we introduce a degradation representation strategy based on contrastive learning to avoid the error amplification problem caused by explicit degradation estimation. Comprehensive experiments show that the proposed algorithm can obtain SR results with excellent visual perception quality on both simulated LR and real-world RSIs. The code is available at https://github.com/hanlinwu/BlindSRSNF.

Deep learning-based algorithms have greatly improved the performance of remote sensing image (RSI) super-resolution (SR). However, increasing network depth and parameters cause a huge burden of computing and storage. Directly reducing the depth or width of existing models results in a large performance drop. We observe that the SR difficulty of different regions in an RSI varies greatly, and existing methods use the same deep network to process all regions in an image, resulting in a waste of computing resources. In addition, existing SR methods generally predefine integer scale factors and cannot perform stepless SR, i.e., a single model can deal with any potential scale factor. Retraining the model on each scale factor wastes considerable computing resources and model storage space. To address the above problems, we propose a saliency-aware dynamic routing network (SalDRN) for lightweight and stepless SR of RSIs. First, we introduce visual saliency as an indicator of region-level SR difficulty and integrate a lightweight saliency detector into the SalDRN to capture pixel-level visual characteristics. Then, we devise a saliency-aware dynamic routing strategy that employs path selection switches to adaptively select feature extraction paths of appropriate depth according to the SR difficulty of subimage patches. Finally, we propose a novel lightweight stepless upsampling module whose core is an implicit feature function for realizing mapping from low-resolution feature space to high-resolution feature space. Comprehensive experiments verify that the SalDRN can achieve a good tradeoff between performance and complexity. The code is available athttps://github.com/hanlinwu/SalDRN.

The mainstream remote sensing image (RSI) super-resolution (SR) algorithms treat tasks with different scale factors independently, and a single model can only process a fixed integer scale factor. However, in practical applications, it is important to continuously super-resolve RSIs to multiple resolutions, as different resolutions present various levels of detail. Retraining the model for each scale factor consumes huge computational resources and storage space. Existing continuous-scale SR models employ static convolutions, and most are designed for natural scenes, ignoring dynamic feature extraction needs for different scale factors and the inherent properties of RSIs. In addition, efficiently obtaining the continuous representation of RSIs and avoiding the artifacts of RSI SR results is still a challenging problem. To address the above problems, we propose a scale-aware dynamic network (SADN) for RSI continuous-scale SR. First, we devise a scale-aware dynamic convolutional (SAD-Conv) layer to handle the strong randomness of the RSI textural distribution and achieve dynamic feature extraction according to scale factors. Second, we devise a continuous-scale upsampling module (CSUM) with the multi-bilinear global implicit function (MBGIF) for any-scale upsampling. The CSUM constructs multiple feature spaces with asymptotic resolutions to approximate the continuous representation of an image, and then, the MBGIF makes full use of multiresolution features to map arbitrary coordinates to spectral values. We evaluate our SADN using various benchmarks, and the experimental results show that the CSUM can efficiently achieve continuous-scale upsampling while maintaining excellent objective and visual performance. Moreover, our SADN uses fewer parameters and even outperforms the state-of-the-art fixed-scale SR methods. The source code is available athttps://github.com/hanlinwu/SADN.

Video super-resolution (VSR), especially continuous-scale VSR, plays a crucial role in improving the quality of satellite video. Continuous-scale VSR aims to use a single model to process arbitrary (integer or non-integer) scale factors, which is conducive to meeting the needs of video images transmission with different compression ratios and arbitrarily zooming by rolling the mouse wheel. In this article, we propose a novel network to achieve continuous-scale satellite VSR (CAVSR). Specifically, first, we propose a time-series-aware dynamic routing deformable alignment module (TDAM) for feature alignment. Second, we develop a scale-adaptive feature extraction module (SFEM), which uses the proposed scale-adaptive convolution (SA-Conv) to dynamically generate different filters based on the input scale information. Finally, we design a global implicit function feature-adaptive walk continuous-scale upsampling module (GFCUM), which can perform feature-adaptive walks according to the input features with different scale information and finally complete the continuous-scale mapping from coordinates to pixel values. Experimental results have demonstrated the CAVSR has superior reconstruction performance.

Conducting research on remote sensing image (RSI) super-resolution (SR) is important, especially in terms of the continuous scale, which is beneficial to the application of RSI, such as RSI object detection and data fusion. Continuous-scale SR aims to use a single model to achieve SR at arbitrary (integer and noninteger) scale factors. Therefore, in this letter, we propose a hierarchical feature aggregation and self-learning network for RSI continuous-scale SR (RSI-HFAS). Our network can magnify the RSI continuously, which is beneficial for extracting the RSI multiscale features. First, we design a hierarchical feature aggregation module (HFAM) that is used for hierarchical feature extraction by placing convolutional layers on different floors and completing global feature fusion, which is crucial for achieving RSI continuous-scale SR with a single model. Second, the proposed network introduces a feedback mechanism, which can refine the hierarchical feature through feature feedback and enrich the texture parts of the RSI step by step. Finally, we design a self-learning upscaling structure to dynamically predict the number and weights of the upsampling filters, which can achieve RSI continuous-scale SR. Compared to the meta-learning based on enhanced deep SR (META-EDSR) method, our experimental results show a nearly 0.2-dB improvement on the metrics of the peak signal-to-noise ratio (PSNR).