Image operators can be instrumental to computational imaging and photography. However, many of them are computationally intensive. In this article, we propose an effective yet efficient joint upsampling method to accelerate various image operators. We show that edge-preserving filtering can be facilitated with a downsampling-and-upsampling process. Moreover, when the extent of smoothing is mild, the process is detail preserving, i.e., the fine details lost in the low-resolution (LR) images can be accurately restored in the high-resolution (HR) images. Given an HR input and an LR output of an operator, we downsample the HR input and calculate its affinities to the HR input. By applying the affinities to the LR output, we promote its resolution. Due to the strong detail-preserving property, the HR output derived in the previous step may exhibit aliasing artifacts around the salient edges. We further refine it based on the linear relations in a small neighborhood to rid the artifacts. Experiments on various image operators show that our method achieves superior quality over the state-of-the-art joint upsampling methods. Furthermore, the running time of our method is linear to the number of pixels. Our naive implementation derives 1080P images in real time (24 fps) on an NVIDIA GTX 3070 GPU.

Smoothing images while preserving salient edges is a crucial task in computational photography. Existing edge-preserving filters suffer from various artifacts, such as halos, gradient reversals, and intensity shifts. Observing that various artifacts are strongly related to salient edges with large gradients, we propose a continuous mapping function to process the gradients. The proposed function is literally edge-preserving, i.e., it keeps large gradients intact while attenuating small gradients. We propose an L1-regularized reconstruction model based on the processed gradients for edge-preserving image filtering. The L1-regularization facilitates the edge-preserving property in the reconstructed results. To solve the proposed L1-regularized model, we implement an efficient algorithm based on the alternating direction method of multipliers (ADMM) and Fourier domain optimization. We have conducted qualitative and quantitative experiments to evaluate the proposed filter. The results demonstrate that our filter better handles various artifacts and delivers superior image quality on various applications. The proposed filter is highly efficient, our GPU implementation takes 70ms to process a color image with 1 megapixel on an NVIDIA GTX 1070 GPU.

Depth-of-field (DoF) rendering is an important technique in computational photography that simulates the human visual attention system. Existing DoF rendering methods usually suffer from a high computational cost. The task of DoF rendering can be accelerated by guided upsampling methods. However, the state-of-the-art guided upsampling methods fail to distinguish the focus and defocus areas, resulting in unsatisfying DoF effects. In this paper, we propose a novel deep weighted guided upsampling network (DWGUN) based on a encoder and decoder framework to jointly upsample the low-resolution DoF image under the guidance of the corresponding high-resolution all-in-focus image. Due to the intuitive weight design, the traditional weighted image upsampling is not tailored to DoF image upsampling. We propose a deep refocus-defocus edge-aware module (DREAM) to learn the spatially-varying weights and embed them in the deep weighted guided upsampling block (DWGUB). We have conducted comprehensive experiments to evaluate the proposed method. Rigorous ablation studies are also conducted to validate the rationality of the proposed components.

Edge-preserving image filtering is an essential task in computational photography and imaging. In this paper, we propose a simple yet effective global edge-preserving filter based on soft clustering, and we propose a novel soft clustering algorithm based on a restricted Gaussian mixture model. Given specified parameters, the soft clustering process is firstly performed on the image to derive the partition matrix, from which the affinity matrix is then constructed for filtering. The filtering output is calculated as the weighted average of the pixels in the local window, so the proposed filter could suppress the intensity shift artifacts that impede most global filters. Besides, the weights in the proposed filter are derived by clustering, which properly separates dissimilar pixels, so the proposed filter could handle the halo artifacts that haunt many local filters. Moreover, our filter provides flexible control over the amount of smoothing that is deficient in the deep learning-based filters. Besides the efficacy in smoothing, the proposed filter naturally has low computational complexity. Qualitative and quantitative results suggest that the proposed filter benefits various applications, including edge-preserving smoothing, image enhancing, flash/non-flash fusion, HDR tone mapping, and dehazing.

Origami has provided a potential way to construct 3D curved structures by folding flat sheet materials without cutting or stretching. As a traditional origami, waterbomb tessellation is widely studied from aspects of science and engineering. However, users cannot easily utilize this kind of origami to fit curved target surfaces because the underlying geometric constraints limit the design space. In this study, we propose computational design methods for approximating cylindrical and axisymmetric curved surfaces based on waterbomb tessellations. With consideration of symmetry and periodic repetition, a single strip of the waterbomb tessellation is first modeled and then longitudinally and circumferentially replicated to construct cylindrical and axisymmetric waterbomb tessellations, respectively. To fulfill flat-foldability, an optimization process is introduced for minimizing flat-foldable residuals iteratively and then a regulation process of the crease pattern is applied for further reducing such residuals. In addition, we demonstrate waterbomb-derivative tessellations with quad-paddings to expand the design variations. Furthermore, rigid-folding sequences and several physically engineered origami pieces are presented. The proposed methods can be utilized to facilitate the design of origami-inspired structures for various engineering design purposes, such as foldable shelters, tubular structures, metamaterials, and so on.