Decomposing a video into a layer-based representation is crucial for easy video editing for the creative industries, as it enables independent editing of specific layers. Existing video-layer decomposition models rely on implicit neural representations (INRs) trained independently for each video, making the process time-consuming when applied to new videos. Noticing this limitation, we propose a meta-learning strategy to learn a generic video decomposition model to speed up the training on new videos. Our model is based on a hypernetwork architecture which, given a video-encoder embedding, generates the parameters for a compact INR-based neural video decomposition model. Our strategy mitigates the problem of single-video overfitting and, importantly, shortens the convergence of video decomposition on new, unseen videos. Our code is available at: https://hypernvd.github.io/.

Instance segmentation is crucial for autonomous driving, but is hindered by the lack of annotated real-world data due to expensive labeling costs. Unsupervised Domain Adaptation (UDA) offers a solution by transferring knowledge from labeled synthetic data to unlabeled real-world data. While UDA methods for synthetic to real-world domains (synth-to-real) excel in tasks such as semantic segmentation and object detection, their application to instance segmentation for autonomous driving remains underexplored and often relies on suboptimal baselines. We introduce UDA4Inst, a powerful framework for synth-to-real UDA in instance segmentation. Our framework enhances instance segmentation through Semantic Category Training and Bidirectional Mixing Training. Semantic Category Training groups semantically related classes for separate training, improving pseudo-label quality and segmentation accuracy. Bidirectional Mixing Training combines instance-wise and patch-wise data mixing, creating coherent composites that enhance generalization across domains. Extensive experiments show UDA4Inst sets a new state-of-the-art on the SYN{###}$THIA \rightarrow$Cityscapes benchmark (mAP 31.3) and introduces results on novel datasets, using UrbanSyn and Synscapes as sources and Cityscapes and KITTI360 as targets. Code and models are available at https://github.com/gyc-code/UDA4Inst.

Advancements in imaging technology have enabled hardware to support 10 to 16 bits per channel, facilitating precise manipulation in applications like image editing and video processing. While deep neural networks promise to recover high bit-depth representations, they still have issues such as banding artifacts, loss of fine texture details, and inadequate preservation of subtle color gradients, due to their reliance on scale-invariant image information, limiting performance in certain scenarios. In this paper, we introduce a novel approach that integrates a super-resolution architecture to extract detailed a priori information from images. By leveraging interpolated data generated during the super-resolution process, our method achieves pixel-level recovery of fine-grained color details. Additionally, we demonstrate that spatial features learned through the super-resolution process significantly contribute to the recovery of detailed color depth information. Experiments on benchmark datasets demonstrate that our approach outperforms state-of-the-art methods, highlighting the potential of super-resolution for high-fidelity color restoration.

Color harmony refers to combinations of colors that look pleasing together. We present a novel strategy to harmonize an image's colors using color-palette manipulation and color naming. Palette-based color manipulation is a method that extracts a few colors to represent the image. Modifying the palette colors modifies the color appearance of the image. A color-naming model is a mechanism to categorize colors into a fixed number of basic color terms. Working from a color-naming model, we derive a set ofprototype colorsand demonstrate that mapping an image's extracted color palette to the nearest prototype colors effectively harmonizes the image's colors. This straightforward approach yields visually compelling, outperforming more complex color harmony methods.

Burst image restoration attempts to effectively utilize the complementary cues appearing in sequential images to produce a high-quality image. Most current methods use all the available images to obtain the reconstructed image. However, using more images for burst restoration is not always the best option regarding reconstruction quality and efficiency, as the images acquired by handheld imaging devices suffer from degradation and misalignment caused by the camera noise and shake. In this paper, we extend the perception-distortion tradeoff theory by introducing multiple-frame information. We propose the area of the unattainable region as a new metric for perception-distortion tradeoff evaluation and comparison. Based on this metric, we analyse the performance of burst restoration from the perspective of the perception-distortion tradeoff under both aligned bursts and misaligned bursts situations. Our analysis reveals the importance of inter-frame alignment for burst restoration and shows that the optimal burst length for the restoration model depends both on the degree of degradation and misalignment.

Self-supervised monocular depth estimation has succeeded in learning scene geometry from only image pairs or sequences. However, it is still highly ill-posed for self-supervised depth estimation to generate high-quality depth maps with both global high accuracy and local fine details. To address this issue, we propose a novel frequency-based recurrent refinement scheme to improve the self-supervised depth estimation. Since the global and local depth representation can be correlated to high/low frequency coefficients in the frequency domain, we propose a frequency-based recurrent depth coefficient refinement (RDCR) scheme, which progressively refines both low frequency and high frequency depth coefficients with an RNN-based architecture in a multi-level manner. During the recurrent process, the depth coefficients generated from the previous time step are used as the input to generate the current depth coefficients, yielding progressively optimized depth estimations. Meanwhile, considering that the depth details often appear in areas with high image frequency, we further improve depth details during the RDCR process by leveraging the image-based high frequency components. Specifically, in each RDCR module, we enhance the high frequency depth representations by selecting and feeding the informative image-based high frequency features with a learned feature weighting mask. Extensive experiments show that the proposed method achieves globally accurate estimation with fine local details, outperforming other self-supervised methods in both quantitative and qualitative comparisons.

Accurate semantic segmentation models typically require significant computational resources, inhibiting their use in practical applications. Recent works rely on well-crafted lightweight models to achieve fast inference. However, these models cannot flexibly adapt to varying accuracy and efficiency requirements. In this paper, we propose a simple but effective slimmable semantic segmentation (SlimSeg) method, which can be executed at different capacities during inference depending on the desired accuracy-efficiency tradeoff. More specifically, we employ parametrized channel slimming by stepwise downward knowledge distillation during training. Motivated by the observation that the differences between segmentation results of each submodel are mainly near the semantic borders, we introduce an additional boundary guided semantic segmentation loss to further improve the performance of each submodel. We show that our proposed SlimSeg with various mainstream networks can produce flexible models that provide dynamic adjustment of computational cost and better performance than independent models. Extensive experiments on semantic segmentation benchmarks, Cityscapes and CamVid, demonstrate the generalization ability of our framework.