Videos captured under real-world adverse weather conditions typically suffer from uncertain hybrid weather efforts. However, existing algorithms can only remove one type of weather degradation at a time and deal with different weather conditions with separate models, thus may fail to handle real-world stochastic hybrid scenarios. Besides, the model training is also infeasible due to the lack of paired video data to characterize the coexistence of multiple weather. To ameliorate the aforementioned issue, we propose a novel unified model, dubbed UniWRV, to remove multiple heterogeneous video weather degradations in an all-in-one fashion. Specifically, to tackle degenerate spatial feature heterogeneity, we propose a tailored weather prior guided module that queries exclusive priors for different instances as prompts to steer spatial feature characterization. To tackle degenerate temporal feature heterogeneity, we propose a dynamic routing aggregation module that can automatically select optimal fusion paths for different instances to dynamically integrate temporal features. Furthermore, we propose a real-world adaptation training scheme that leverages CLIP priors to provide semantic supervision for unlabeled real-world weather-degraded videos, thereby enabling the model to better cope with the diverse and complex real-world weather conditions. Additionally, we managed to construct a new synthetic video dataset, termed HWVideo, for learning and benchmarking multiple hybrid adverse weather removal, which contains 15 hybrid weather conditions with a total of 1500 adverse-weather/clean paired video clips. Real-world hybrid weather videos are also collected to facilitate model generalizability. Comprehensive experiments demonstrate that our UniWRV exhibits robust and superior adaptation capability in multiple heterogeneous degradations learning scenarios, including various generic video restoration tasks beyond weather removal.

In this paper, we aim ambitiously for a realistic yet challenging problem, namely, how to reconstruct high-quality 3D scenes from sparse low-resolution views that simultaneously suffer from deficient perspectives and clarity. Whereas existing methods only deal with either sparse views or low-resolution observations, they fail to handle such hybrid and complicated scenarios. To this end, we propose a novel Sparse-view Super-resolution 3D Gaussian Splatting framework, dubbed S2Gaussian, that can reconstruct structure-accurate and detail-faithful 3D scenes with only sparse and low-resolution views. The S2Gaussian operates in a two-stage fashion. In the first stage, we initially optimize a low-resolution Gaussian representation with depth regularization and densify it to initialize the high-resolution Gaussians through a tailored Gaussian Shuffle Split operation. In the second stage, we refine the high-resolution Gaussians with the super-resolved images generated from both original sparse views and pseudo-views rendered by the low-resolution Gaussians. In which a customized blur-free inconsistency modeling scheme and a 3D robust optimization strategy are elaborately designed to mitigate multi-view inconsistency and eliminate erroneous updates caused by imperfect supervision. Extensive experiments demonstrate superior results and in particular establishing new state-of-the-art performances with more consistent geometry and finer details. Project Page https://jeasco.github.io/S2Gaussian/.

Despite substantial progress, the existing all-in-one image restoration methods still lack the ability to adaptively sense and accurately represent degradation information, thus hindering the enhancement of restoration performance. In addition, due to the large uncertainty and fuzziness of the data distribution in real scenarios compared to the training data, the model's generalization ability is often limited. To address the above issues, we propose a novel adaptive fuzzy degradation perception approach based on fuzzy theory that includes two tactics: 1) Fuzzy Degradation Perceiver (FDP); and 2) Test-time Self-supervised Prompt Fine-tuning (TSPF). On the one hand, we introduce the FDP, which leverages the rich visual language prior knowledge in CLIP to learn the prompt representations of different degradations. These prompts are regarded as semantic representations of various degradation fuzzy sets, achieving adaptive degradation perception by computing the degrees of membership between input images and the fuzzy sets. On the other hand, we propose the TSPF strategy, which is capable of self-supervised optimization of degraded fuzzy sets according to real-world scenarios during testing. This strategy improves the model's ability to perceive and represent the degraded information in data with real-world distributions. Thanks to the above key strategies, our method significantly improves degradation perception capability and image restoration quality while exhibiting excellent generalization in complex real-world scenarios. Extensive experiments on multiple benchmark datasets confirm that our approach achieves state-of-the-art performance in all-in-one image restoration.

Semantic segmentation of remote sensing (RS) images refers to labeling each pixel with a class to identify objects or land cover types. Existing mainstream spatial-domain semantic segmentation methods are mainly categorized into convolutional neural network (CNN)-based and vision transformer (ViT)-based approaches. The former excels at capturing local features, while the latter is adept at extracting global features. Several recent approaches consider combining CNN and ViT to efficiently capture local and global features. However, these approaches still struggle to capture complete features of the RS images, resulting in inaccurate segmentation. To address this issue, we introduce the fast Fourier transform (FFT), which transforms images into the frequency domain for feature extraction, acquiring the image-size receptive field that can complement spatial-domain methods. Based on this, we propose a boundary-aware spatial and frequency dual-domain transformer, termed dual-domain transformer. Specifically, our dual-domain transformer incorporates a dual-domain mixer (DualM), where the spatial-domain branch combines depthwise convolution and the attention mechanism to extract local and global features effectively, while the frequency-domain branch uses FFT to extract image-size features. The two branches complement each other, enabling a more comprehensive feature extraction of RS images. Meanwhile, a boundary-guided training strategy utilizing a boundary-aware module (BAM) is devised to constrain the model extract and predict boundary detail texture, which is an auxiliary task. In addition, the decoder incorporates a scale-feature fusion module (SFM) for adaptive information fusion between the encoder and decoder. Comprehensive experiments on the Zeebrugge and ISPRS datasets, including Vaihingen and Potsdam, showcase that the dual-domain transformer significantly outperforms state-of-the-art (SOTA) methods.

For the sake of eliminating multiple degradations, most existing multitask image restoration methods prefer to learn the properties of each degradation type, which is often accompanied by a bloated model size and a heavy learning burden. To tackle the aforementioned issues, in this article, we propose to treat multiple degradations uniformly to achieve degradation type-agnostic multitask image restoration. We observe that the degradations in different spatial locations are always morphologically similar while the background sceneries vary greatly. In accordance with the aforementioned observation, we decouple the degradation features and the background features by an efficient fuzzy clustering method. The degradation features contain all the diverse degradation information, while the images are recovered from the decoupled background features. In practice, we discover a uniformity between the fuzzy C-means algorithm and cross attention and propose a deep fuzzy clustering transformer to achieve degradation type-agnostic background extraction via feature map clustering based on spatial distribution characteristics. Furthermore, to capture the spatial distribution properties of an image, an efficient global attention tree (GAT) is devised to provide a global spatial receptive field for the clustering process. By virtue of the quadtree structure, the proposed GATs enable more efficient global modeling than existing methods. Our experimental analysis showed that the proposed method outperformed the state-of-the-art models in terms of both efficiency and performance.