Deep neural networks have significantly advanced video denoising. However, deploying accurate denoisers across platforms remains challenging, particularly under real-time constraints and with high-resolution inputs. While explicit motion estimation modules improve temporal alignment and enhance denoising quality, they are often omitted in lightweight models due to their computational burden. This paper investigates the trade-off between denoising quality and computational efficiency when using spatially coarse optical flow, and shows that reduced-resolution flow retains most temporal denoising benefits while significantly lowering computational cost. In addition, we introduce an inference-free imbalanced noise pre-training strategy that encourages effective temporal aggregation and complements coarse alignment without modifying the runtime architecture. The proposedLightweightVideoDenoiser (LVD) achieves state-of-the-art performance among lightweight models on RAW video denoising benchmarks while maintaining low latency suitable for real-time edge deployment. Our code is available athttps://github.com/camalab-ai/LVD

We introduce Frame Interpolation Pre-training (FIP), a simple learning technique for lifting deep image denoisers to video denoising with improved implicit temporal alignment. Modern video denoising networks typically rely on explicit motion estimation and alignment which are computationally intensive and harder to re-design and re-train, restricting their application scope and usability. Conversely, stacking frames and image denoisers, without incorporating explicit motion estimation modules, improves speed and benefits from a simpler design, thereby facilitating their generalizability to the video domain. However, it leads to lower accuracy due to suboptimal capture of temporal dependencies. To better leverage the adjacent frames in this setting and reduce the accuracy gap, we propose a novel training regime that divides the standard supervised training of the denoising task into two phases. In the initial phase, FIP guides the network to interpolate a fully masked central frame using only adjacent noisy input frames. In the subsequent phase, the pre-trained network is fine-tuned on denoising the central frame, now using all noisy input frames. Extensive diagnostics indicate that FIP-based networks provide better implicit motion estimation and temporal alignment. In effect, qualitative and quantitative evaluation on standard video denoising datasets with synthetic and real noise demonstrates that FIP consistently improves video denoising accuracy of motion-aware, video-lifted image denoisers without additional computational overhead during training and test time. Our code is available at https://github.com/camalab-ai/FIP.

Optical flow is integral to numerous video processing tasks such as stabilization. Although recent advancements in optical flow estimation have shown significant efficacy in general scenes, their applicability to challenging medical scenarios, which exhibit unique domain-specific visual phenomena, remains limited. Supervised learning methods facilitate the robust training of motion estimators. However, the absence of ground truth optical flow in many medical video-assisted applications poses a significant barrier to their progress. This is particularly evident in Video-Assisted Dentistry (VAD), where enhanced and continual vision could improve the educational, training, and fully operational dental workflows. Addressing this gap, we propose a new method for optical flow adaptation on real videos aided by a large dataset of synthetic dental videos with corresponding ground truth optical flows. Notably, the method employs a novel no-reference evaluation measure for unsupervised learning and incrementally fine-tunes optical flow neural networks. It facilitates their transition from synthetic to real dynamic dental scenes characterized by variable illumination, specular reflections, fluid dynamics, and sparse textures. In effect, this study demonstrates the effective streamed adaptation of multiple state-of-the-art optical flow networks to real medical scenes of video-assisted conservative dental treatments. Our code is available at https://github.com/camalab-ai/sofa-flow.

Infertility affects around 15% of couples worldwide. Male fertility problems include poor sperm quality and low sperm count. The advanced fertility treatment methods like ICSI are nowadays supported by vision systems to assist embryologists in selecting good quality sperm. Computer-Assisted Semen Analysis (CASA) provides quantitative and qualitative sperm analysis concerning concentration, motility, morphology, vitality, and fragmentation. However, fertility assessment algorithms often neglect individual spermatozoon tail and its beating patterns because recognizing the tails in blurry microscopic images reliably is challenging. In this article, we propose that models trained with head and tail part classes can better localize parts and segment the whole spermatozoon objects. Usually, the training of segmentation sperm models is supported by image-level augmentation. We argue that models guided by class-specific data augmentation attend to less discriminative sperm parts. To demonstrate this, we decouple the augmentation into object-level and background augmentation for the sperm part segmentation problem. Our proposed method outperforms state-of-the-art methods on the SegSperm dataset. Moreover, our ablation studies confirm the effectiveness of the proposed part-based object representation and augmentation.

(1) About 15-20% of couples struggle with the problem of infertility. 30 to 40% of these cases are caused by abnormalities in the structure and motility of sperm. Sometimes the only possibility for such people is to use the procedure of artificial insemination. CASA systems are used to increase the efficiency of this procedure by selecting the appropriate sperm cell. (2) This paper presents an approach to the sperm classification on the basis of its entire structure analysis, including flagella - often poorly visible and therefore ignored in the CASA systems element. The training of the Mask R-CNN architecture was performed on 2 publicly available and one specially created for this purpose sperm database. A 14-element feature vector was also proposed for the classification of 4 classes of typical head defects (amorphous, normal, tapered and pyriform) by the Support Vector Machine. (3) The sperm head (mAP 94.28%) and the whole flagellum (mAP 90.29%) were successfully detected. However, the flagella segmentation results were significantly lower (50.88%) than that the head segmentation (88.32%). Classification with SVM scored 82% accuracy. (4) Research has shown that segmentation and the use of a simple SVM classifier allow for quite good results in the classification of sperm defects. However, it is important to develop a larger whole sperm database, to improve the segmentation results.

Denoising videos in real-time is critical in many applications, including robotics and medicine, where varying-light conditions, miniaturized sensors, and optics can substantially compromise image quality. This work proposes the first video denoising method based on a deep neural network that achieves state-of-the-art performance on dynamic scenes while running in real-time on VGA video resolution with no frame latency. The backbone of our method is a novel, remarkably simple, temporal network of cascaded blocks with forward block output propagation. We train our architecture with short, long, and global residual connections by minimizing the restoration loss of pairs of frames, leading to a more effective training across noise levels. It is robust to heavy noise following Poisson-Gaussian noise statistics. The algorithm is evaluated on RAW and RGB data. We propose a denoising algorithm that requires no future frames to denoise a current frame, reducing its latency considerably. The visual and quantitative results show that our algorithm achieves state-of-the-art performance among efficient algorithms, achieving from two-fold to two-orders-of-magnitude speed-ups on standard benchmarks for video denoising.

In dentistry, microscopes have become indispensable optical devices for high-quality treatment and micro-invasive surgery, especially in the field of endodontics. Recent machine vision advances enable more advanced, real-time applications including but not limited to dental video deblurring and workflow analysis through relevant metadata obtained by instrument motion trajectories. To this end, the proposed work addresses dental video deblurring and instrument segmentation in a Multi-task Learning fashion, leveraging spatio-temporal adaptive kernels via a recurrent design. The task-specific branches of our architecture employ the responses of those kernels to recover sharper video frames and yield the dental instrument segmentation mask. We demonstrate that the proposed method improves deblurring while retaining segmentation performance under a low computational footprint.

We study a variational formulation for reconstructing nonlinearly distorted signals corrupted with a Poisson-Gaussian noise. In this situation, the data fidelity term consists of a sum of a weighted least squares term and a logarithmic one. Both of them are precomposed by a nonlinearity, modelling a clipping effect, which is assumed to be rational. A regularization term, being a piecewise rational approximation of the ℓ0function provides a suitable sparsity measure with respect to a preset linear operator. We propose a global optimization approach for such a problem. More specifically, it is first transformed into a generalized moment problem by introducing some auxiliary variables. Then, a hierarchy of semidefinite programming relaxations is built. Numerical examples show the good performance of the proposed approach.

Angiographic imaging is a crucial domain of medical imaging. In particular, Magnetic Resonance Angiography (MRA) is used for both clinical and research purposes. This article presents the first framework geared toward the design of virtual MRA images from real MRA images. It relies on a pipeline that involves image processing, vascular modeling, computational fluid dynamics and MR image simulation, with several purposes. It aims to provide to the whole scientific community (1) software tools for MRA analysis and blood flow simulation; and (2) data (computational meshes, virtual MRAs with associated ground truth), in an open-source/open-data paradigm. Beyond these purposes, it constitutes a versatile tool for progressing in the understanding of vascular networks, especially in the brain, and the associated imaging technologies. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The Poisson--Gaussian model can accurately describe the noise present in a number of imaging systems. However most existing restoration methods rely on approximations of the Poisson--Gaussian noise statistics. We propose a convex optimization strategy for the reconstruction of images degraded by a linear operator and corrupted with a mixed Poisson--Gaussian noise. The originality of our approach consists of considering the exact, mixed continuous-discrete model corresponding to the data statistics. After establishing the Lipschitz differentiability and convexity of the Poisson--Gaussian neg-log-likelihood, we derive a primal-dual iterative scheme for minimizing the associated penalized criterion. The proposed method is applicable to a large choice of convex penalty terms. The robustness of our scheme allows us to handle computational difficulties due to infinite sums arising from the computation of the gradient of the criterion. We propose finite bounds for these sums, that are dependent on the current image estimate, and thus adapted to each iteration of our algorithm. The proposed approach is validated on image restoration examples. Then, the exact data fidelity term is used as a reference for studying some of its various approximations. We show that in a variational framework the shifted Poisson and exponential approximations lead to very good restoration results.

International audience

This paper deals with noise parameter estimation from a single image under Poisson-Gaussian noise statistics. The problem is formulated within a mixed discrete-continuous optimization framework. The proposed approach jointly estimates the signal of interest and the noise parameters. This is achieved by introducing an adjustable regularization term inside an optimized criterion, together with a data fidelity error measure. The optimal solution is sought iteratively by alternating the minimization of a label field and of a noise parameter vector. Noise parameters are updated at each iteration using an Expectation-Maximization approach. The proposed algorithm is inspired from a spatial regularization approach for vector quantization. We illustrate the usefulness of our approach on macroconfocal images. The identified noise parameters are applied to a denoising algorithm, so yielding a complete denoising scheme.

In this work, we consider a class of differentiable criteria for sparse image computing problems, where a nonconvex regularization is applied to an arbitrary linear transform of the target image. As special cases, it includes edge-preserving measures or frame-analysis potentials commonly used in image processing. As shown by our asymptotic results, the $\ell_2-\ell_0$ penalties we consider may be employed to provide approximate solutions to $\ell_0$-penalized optimization problems. One of the advantages of the proposed approach is that it allows us to derive an efficient majorize-minimize subspace algorithm. The convergence of the algorithm is investigated by using recent results in nonconvex optimization. The fast convergence properties of the proposed optimization method are illustrated through image processing examples. In particular, its effectiveness is demonstrated on several data recovery problems.

Haar-like features are ubiquitous in computer vision, e.g. for Viola and Jones face detection or local descriptors such as Speeded-Up-Robust-Features. They are classically computed in one pass over integral image by reading the values at the feature corners. Here we present a new, general parsing formalism for convolving them more efficiently. Our method is fully automatic and applicable to an arbitrary set of Haar-like features. The parser reduces the number of memory accesses which are the main computational bottleneck during convolution on modern computer architectures. It first splits the features into simpler kernels. Then it aligns and reuses them where applicable forming an ensemble of recursive convolution trees, which can be computed faster. This is illustrated with experiments, which show a significant speed-up over the classic approach.

A Poisson-Gaussian model accurately describes the noise present in many imaging systems such as CCD cameras or fluorescence microscopy. However most existing restoration strategies rely on approximations of the Poisson-Gaussian noise statistics. We propose a convex optimization algorithm for the reconstruction of signals degraded by a linear operator and corrupted with mixed Poisson-Gaussian noise. The originality of our approach consists of considering the exact continuous-discrete model corresponding to the data statistics. After establishing the Lipschitz differentiability of the Poisson-Gaussian log-likelihood, we derive a primal-dual iterative scheme for minimizing the associated penalized criterion. The proposed method is applicable to a large choice of penalty terms. The robustness of our scheme allows us to handle computational difficulties due to infinite sums arising from the computation of the gradient of the criterion. The proposed approach is validated on image restoration examples.

In this paper, we consider a class of differentiable criteria for sparse image recovery problems. The regularization is applied to a linear transform of the target image. As special cases, it includes edge preserving measures or frame analysis potentials. As shown by our asymptotic results, the considered ℓ2- ℓ0penalties may be employed to approximate solutions to ℓ0penalized optimization problems. One of the advantages of the approach is that it allows us to derive an efficient Majorize-Minimize Memory Gradient algorithm. The fast convergence properties of the proposed optimization algorithm are illustrated through image restoration examples.

Quantization, defined as the act of attributing a finite number of grey-levels to an image, is an essential task in image acquisition and coding. It is also intricately linked to various image analysis tasks, such as denoising and segmentation. In this paper, we investigate quantization combined with regularity constraints, a little-studied area which is of interest, in particular, when quantizing in the presence of noise or other acquisition artifacts. We present an optimization approach to the problem involving a novel two-step, iterative, flexible, joint quantizing-regularization method featuring both convex and combinatorial optimization techniques. We show that when using a small number of grey-levels, our approach can yield better quality images in terms of SNR, with lower entropy, than conventional optimal quantization methods.