Motion blur of fast-moving subjects is a longstanding problem in photography and very common on mobile phones due to limited light collection efficiency, particularly in low-light conditions. While we have witnessed great progress in image deblurring in recent years, most methods require significant computational power and have limitations in processing high-resolution photos with severe local motions. To this end, we develop a novel face deblurring system based on the dual camera fusion technique for mobile phones. The system detects subject motion to dynamically enable a reference camera, e.g., ultrawide angle camera commonly available on recent premium phones, and captures an auxiliary photo with faster shutter settings. While the main shot is low noise but blurry (Figure 1(a)), the reference shot is sharp but noisy (Figure 1(b)). We learn ML models to align and fuse these two shots and output a clear photo without motion blur (Figure 1(c)). Our algorithm runs efficiently on Google Pixel 6, which takes 463 ms overhead per shot. Our experiments demonstrate the advantage and robustness of our system against alternative single-image, multi-frame, face-specific, and video deblurring algorithms as well as commercial products. To the best of our knowledge, our work is the first mobile solution for face motion deblurring that works reliably and robustly over thousands of images in diverse motion and lighting conditions.

We present an algorithm for real-time video denoising on mobile platforms. Based on Gaussian-Laplacian pyramid decomposition, our solution's main contributions are fast alignment and a new interpolation function that fuses noisy frames into a denoised result. The interpolation function is adaptive to local and global properties of the input frame, robust to motion alignment errors, and can be computed efficiently. We show that the proposed algorithm has comparable quality to offline high-quality video denoising methods, but is orders of magnitude faster. On a modern mobile platform, our work takes less than 20ms to process one HD frame, and it achieves the highest score on a public benchmark.

This article describes Pan360, a 360 X 180 panorama capturing and viewing product developed and launched by our team, and our demo plan.