Denoising is one of the fundamental steps of the processing pipeline that converts data captured by a camera sensor into a display-ready image or video. It is generally performed early in the pipeline, usually before demosaicking, although studies swapping their order or even conducting them jointly have been proposed. With the advent of deep learning, the quality of denoising algorithms has steadily increased. Even so, modern neural networks still have a hard time adapting to new noise levels and scenes, which is indispensable for real-world applications. With those in mind, we propose a self-similarity-based denoising scheme that weights both a pre- and a post-demosaicking denoiser for Bayer-patterned CFA video data. We show that a balance between the two leads to better image quality, and we empirically find that higher noise levels benefit from a higher influence pre-demosaicking. We also integrate temporal trajectory prefiltering steps before each denoiser, which further improve texture reconstruction. The proposed method only requires an estimation of the noise model at the sensor, accurately adapts to any noise level, and is competitive with the state of the art, making it suitable for real-world videography.

The increased interest in consumer-grade high dynamic range (HDR) images and videos in recent years has caused a proliferation of HDR deghosting algorithms. Despite numerous proposals, a fast, memory-efficient, and robust algorithm has been difficult to achieve. This paper addresses this problem by leveraging the power of attention and U-Net-based neural representations and using a conservative deghosting strategy. Given two bracketed exposures of a scene, we produce an HDR image that maximally resembles the high exposure where it is well-exposed and fuses aligned information from both exposures otherwise. We evaluate the performance of our algorithm under several different challenging scenarios, using both visual and quantitative results, and show that it matches the state-of-the-art algorithms despite using only two exposures and has significantly lower computational complexity. Furthermore, the parameters of our algorithm greatly simplify deploying its different versions for devices with a variety of computational constraints, including mobile devices.

Denoising and demosaicing are two critical components of the image/video processing pipeline. While historically these two tasks have mainly been considered separately, current neural network approaches allow to obtain state-of-the-art results by treating them jointly. However, most existing research focuses in single image or burst joint denoising and demosaicing (JDD). Although related to burst JDD, video JDD deserves its own treatment. In this work we present an empirical exploration of different design aspects of video joint denoising and demosaicing using neural networks. We compare recurrent and non-recurrent approaches and explore aspects such as type of propagated information in recurrent networks, motion compensation, video stabilization, and network architecture. We found that recurrent networks with motion compensation achieve best results. Our work should serve as a strong baseline for future research in video JDD.