Droplet-based microfluidic devices, with their high throughput and low power consumption, have found wide-ranging applications in the life sciences, such as drug discovery and cancer detection. However, the lack of real-time methods for accurately estimating droplet generation parameters has resulted in droplet microfluidic systems remaining largely offline-controlled, making it challenging to achieve efficient feedback in droplet generation. To meet the real-time requirements, it's imperative to minimize the data throughput of the collection system while employing parameter estimation algorithms that are both resource-efficient and highly effective. Spike camera, as an innovative form of neuromorphic camera, facilitates high temporal resolution scene capture with comparatively low data throughput. In this paper, we propose a real-time evaluation method for high-speed droplet parameters based on spike-based microfluidic flow-focusing, named RTDE, that integrates spike camera into the droplet collection system to efficiently capture information using spike stream. To process the spike stream effectively, we develop a spike-based estimation algorithm for real-time droplet generation parameters. To validate the performance of our method, we collected spike-based droplet datasets (SDD), comprising synthetic and real data with varying flow velocities, frequencies, and droplet sizes. Experiments result on these datasets consistently demonstrate that our method achieves parameter estimations that closely match the ground truth values, showcasing high precision. Furthermore, comparative experiments with image-based parameter estimation methods highlight the superior time efficiency of our method, enabling real-time calculation of parameter estimations. Cdoe and datasets are avaliable at: https://github.com/Onetism/RTDE

Recent works have made great progress in semantic segmentation by exploiting richer context, most of which are designed from a spatial perspective. In contrast to previous works, we present the concept of class center which extracts the global context from a categorical perspective. This class-level context describes the overall representation of each class in an image. We further propose a novel module, named Attentional Class Feature (ACF) module, to calculate and adaptively combine different class centers according to each pixel. Based on the ACF module, we introduce a coarse-to-fine segmentation network, called Attentional Class Feature Network (ACFNet), which can be composed of an ACF module and any off-the-shell segmentation network (base network). In this paper, we use two types of base networks to evaluate the effectiveness of ACFNet. We achieve new state-of-the-art performance of 81.85% mIoU on Cityscapes dataset with only finely annotated data used for training.

Because of the subtle structure of focused plenoptic camera, the exact geometry parameters cannot be retrieved, which leads to inaccurate f-number matching or refocusing errors in light field processing. In this paper, a novel blind calibration method is proposed to calculate the geometry parameters for the focused plenoptic cameras. The blind calibration model is derived based on the geometry projection analysis to establish the relationship between the patch-size of each micro-image used in subaperture image rendering and geometry parameters. A triple-level calibration board is designed to realize calibration via single shot. A gradient-SSIM-based fractional-pixel matching is proposed to retrieve the precise rendering patch-size for the calibration model. Experimental results demonstrate that the proposed method is robust to different focused plenoptic cameras and can get the geometry parameters with high accuracy.

Understanding the image formation inside plenoptic cameras is significant for the investigations of improving the low spatial resolution. Most researches explore the image formation from the perspective of geometric optics. However, as the hardware components in combination with low-aperture optical systems become smaller and smaller, geometric analysis will no longer be valid due to diffraction effects. In this paper, a wave-optic-based model is proposed that uses the Fresnel diffraction equation to propagate the whole object field into the plenoptic systems. The proposed model employs averaging of intensities on the sensor from uncorrelated coherent wave to avoid interference during propagations among the optical component planes. Besides, by utilizing the method of multiple partial propagations, the proposed model is much flexible at sampling on propagation planes. In order to verify the effectiveness of the proposed model, numerical simulations are conducted by comparing with existing wave optic model under different optical configurations of plenoptic cameras. Results demonstrate that the proposed model can describe the light field image formation properly. In addition, the time for image formation has been reduced by a factor of 19.22 using the proposed model.

In this paper, a novel light field (LF) stitching method is proposed to handle parallax more flexibly and accurately. The depth guided feature point filtering and depth-based motion model are proposed to warp the 4D meshes in the LFs adapting to the relative depth layer distance between the mesh center and the feature points. Also, 4D graph-cut is applied in LF fusion to reduce ghosting effects and produce better refocusing effects. Experimental results show that the proposed method works well without producing visual artifacts like ghosting or misalignments. It outperforms the existing approaches obviously in terms of the subjective quality of the stitched LFs, RMSE and the coherence in epipolar plane image (EPI).

Conventional depth estimation methods are confined by the occluder and homogenous regions in the scene. In this paper, we propose a new depth estimation and enhancement method. The raw depth is calculated from analyzing the Consistency Metric Range (CMR) in the angular patch. Confident depth map is obtained by analyzing the variation of CMR within a neighborhood around the lowest CMR curves. Confident depth points are propagated to the whole image by global optimization with weighted neighborhood smoothness, gradient and second derivative constraints. Finally, depth is enhanced by using weighted median filter. The experimental results demonstrated the effectiveness of the proposed approach in providing much clearer transitions of texture regions and much smoother homogenous regions after depth propagation.