Welcome to the 16th IEEE International Conference on Computational Photography (ICCP 2024), taking place at the Ecole Polytéchnique Fédérale (EPFL) in Lausanne, Switzerland! This is the first time that ICCP takes place in Continental Europe, specifically on the shore of beautiful Lake Geneva. This year's two-and-a-half day conference features 18 accepted papers, 3 keynote talks, 8 invited talks, and 50 posters and/or demos.

We report Tensorial Tomographic Differential Phase-Contrast microscopy (T2DPC), a quantitative label-free tomographic imaging method for simultaneous measurement of phase and anisotropy. T2DPC extends differential phase-contrast microscopy, a quantitative phase imaging technique, to highlight the vectorial nature of light. The method solves for permittivity tensor of anisotropic samples from intensity measurements acquired with a standard microscope equipped with an LED matrix, a circular polarizer, and a polarization-sensitive camera. We demonstrate accurate volumetric reconstructions of refractive index, birefringence, and orientation for various validation samples, and show that the reconstructed polarization structures of a biological specimen are predictive of pathology.

Segmentation and reconstruction of arteries is important for a variety of medical and engineering fields, such as surgical planning and physiological modeling. However, manual methods can be laborious and subject to a high degree of human variability. In this work, we developed various convolutional neural network ( CNN ) architectures to segment Stanford type B aortic dissections ( TBADs ), characterized by a tear in the descending aortic wall creating a normal channel of blood flow called a true lumen and a pathologic channel within the wall called a false lumen. We introduced several variations to the two-dimensional ( 2D ) and three-dimensional (3 D ) U-Net, where small stacks of slices were inputted into the networks instead of individual slices or whole geometries. We compared these variations with a variety of CNN segmentation architectures and found that stacking the input data slices in the upward direction with 2D U-Net improved segmentation accuracy, as measured by the Dice similarity coefficient ( DC ) and point-by-point average distance ( AVD ), by more than 15\% . Our optimal architecture produced DC scores of 0.94, 0.88, and 0.90 and AVD values of 0.074, 0.22, and 0.11 in the whole aorta, true lumen, and false lumen, respectively. Altogether, the predicted reconstructions closely matched manual reconstructions.

We present a feature-free photogrammetric technique that enables quantitative 3D mesoscopic (mm-scale height variation) imaging with tens-of-micron accuracy from sequences of images acquired by a smartphone at close range (several cm) under freehand motion without additional hardware. Our end-to-end, pixel-intensity-based approach jointly registers and stitches all the images by estimating a coaligned height map, which acts as a pixel-wise radial deformation field that orthorectifies each camera image to allow plane-plus-parallax registration. The height maps themselves are reparameterized as the output of an untrained encoder-decoder convolutional neural network (CNN) with the raw camera images as the input, which effectively removes many reconstruction artifacts. Our method also jointly estimates both the camera’s dynamic 6D pose and its distortion using a nonparametric model, the latter of which is especially important in mesoscopic applications when using cameras not designed for imaging at short working distances, such as smartphone cameras. We also propose strategies for reducing computation time and memory, applicable to other multi-frame registration problems. Finally, we demonstrate our method using sequences of multi-megapixel images captured by an un-stabilized smartphone on a variety of samples (e.g., painting brushstrokes, circuit board, seeds).

This paper introduces a new method of data-driven microscope design for virtual fluorescence microscopy. We use a deep neural network (DNN) to effectively design optical patterns for specimen illumination that substantially improve upon the ability to infer fluorescence image information from unstained microscope images. To achieve this design, we include an illumination model within the DNN’s first layers that is jointly optimized during network training. We validated our method on two different experimental setups, with different magnifications and sample types, to show a consistent improvement in performance as compared to conventional microscope imaging methods. Additionally, to understand the importance of learned illumination on the inference task, we varied the number of illumination patterns being optimized (and thus the number of unique images captured) and analyzed how the structure of the patterns changed as their number increased. This work demonstrates the power of programmable optical elements at enabling better machine learning algorithm performance and at providing physical insight into next generation of machine-controlled imaging systems.

One primary technical challenge in photoacoustic microscopy (PAM) is the necessary compromise between spatial resolution and imaging speed. In this study, we propose a novel application of deep learning principles to reconstruct undersampled PAM images and transcend the trade-off between spatial resolution and imaging speed. We compared various convolutional neural network (CNN) architectures, and selected a Fully Dense U-net (FD U-net) model that produced the best results. To mimic various undersampling conditions in practice, we artificially downsampled fully-sampled PAM images of mouse brain vasculature at different ratios. This allowed us to not only definitively establish the ground truth, but also train and test our deep learning model at various imaging conditions. Our results and numerical analysis have collectively demonstrated the robust performance of our model to reconstruct PAM images with as few as 2% of the original pixels, which can effectively shorten the imaging time without substantially sacrificing the image quality.

Recent machine learning techniques have dramatically changed how we process digital images. However, the way in which we capture images is still largely driven by human intuition and experience. This restriction is in part due to the many available degrees of freedom that alter the image acquisition process (lens focus, exposure, filtering, etc). Here we focus on one such degree of freedom - illumination within a microscope - which can drastically alter information captured by the image sensor. We present a reinforcement learning system that adaptively explores optimal patterns to illuminate specimens for immediate classification. The agent uses a recurrent latent space to encode a large set of variably-illuminated samples and illumination patterns. We train our agent using a reward that balances classification confidence with image acquisition cost. By synthesizing knowledge over multiple snapshots, the agent can classify on the basis of all previous images with higher accuracy than from naively illuminated images, thus demonstrating a smarter way to physically capture task-specific information.

In this work we provide an introduction to the Wigner Distribution Function (WDF) using geometric optics principles. The WDF provides a useful model of wave-fields, allowing simulation of diffraction and interference effects. We attempt to explain these Fourier optics concepts to computational photography researchers by clarifying the relationship between the WDF and position-angle representations. We demonstrate how the WDF can be used to simulate diffraction effects using a light field representation and discuss its validity in the near-field, far-field, and under the paraxial approximation. Finally, we demonstrate that although the WDF representation contains negative values, any projection always yields a non-negative intensity value.

We present a photographic method to enhance intensity differences between objects at varying distances from the focal plane. By combining a unique capture procedure with simple image processing techniques, the detected brightness of an object is decreased proportional to its degree of defocus. A camera-projector system casts distinct grid patterns onto a scene to generate a spatial distribution of point reflections. These point reflections relay a relative measure of defocus that is utilized in postprocessing to generate a highlighted DOF photograph. Trade-offs between three different projector-processing pairs are analyzed, and a model is developed to help describe a new intensity-dependent depth of field that is controlled by the pattern of illumination. Results are presented for a primary single snapshot design as well as a scanning method and a comparison method. As an application, automatic matting results are presented.