Quanta image sensors are an emerging type of image sensor offering single-photon sensitivity. In this paper, we propose a deep unfolding-based image reconstruction method that integrates an alternating direction method of multipliers optimization with a total variation prior. The proposed method effectively combines model-based approaches with deep learning, providing enhanced interpretability and performance. Experimental results demonstrate that our approach outperforms conventional methods based on the observation model for quanta image sensors.

With the advancement of quantum computing, quantum image representations (QIRs) have been widely studied as platforms for quantum image processing. While previous studies have focused on theoretical qubit counts and circuit complexities, there has been limited quantitative evaluation of image reconstruction performance from quantum states. In this study, we clarify the theoretical upper bounds of restoration performance for three representative QIRs: qubit lattice (QL), flexible representation of quantum images (FRQI), and novel enhanced quantum representation (NEQR). Also, we validate them through large-scale GPU-based simulations using cuQuantum. Specifically, we (1) derive a prediction formula for PSNR in probabilistic schemes like QL and FRQI, independent of image content and size; (2) show that deterministic schemes like NEQR require a number of samples following a harmonic number for complete restoration; and (3) perform large-scale simulations to verify these findings. These results provide fundamental insights for designing QIRs and their applications on NISQ devices.

Heart rate (HR) is a key parameter in evaluating the physiological and emotional states of a person. In this paper, we propose a novel video-based heart rate (HR) estimation method based on physiological modeling with multispectral imaging. To capture blood volume pulse (BVP) associated with a person’s heartbeat, we utilize a camera that records multispectral video consisting of red, green, blue, and near-infrared information. The novelty of the proposed method is the incorporation of a physiological BVP model into a multispectral HR estimation framework. The integration of a physiological model-based BVP signal extraction scheme into an adaptive multispectral framework enables the suppression of noise derived from ambient light and the accurate extraction of the BVP signal, thereby enhancing HR estimation performance. The experiments using RGB/NIR video datasets demonstrate the effectiveness of the proposed method.

Quanta image sensors are a novel paradigm in image sensor technology. Their direct application to quanta image sensors-based imaging systems is challenging because a bit-plane image is a set of binary images. In this paper, we introduce spatio-temporal priors based on the intensity invariance and smoothness characteristics of the motion vector. Specifically, we model when the image sequences align with the correct motion vector, the spatiotemporal structure becomes more consistent. Moreover, the spatial smoothness prior is incorporated through the smoothing filtering of the evaluation metrics of motion vector candidates. The experimental results show that the proposed method is more effective than conventional methods.

Lookup tables (LUTs) are commonly used to speed up image processing by handling complex mathematical functions like sine and exponential calculations. They are used in various applications such as camera image processing, high-dynamic range imaging, and edge-preserving filtering. However, due to the increasing gap between computing and input/output performance, LUTs are becoming less effective. Even though specific circuits like SIMD can improve LUT efficiency, they still need to bridge the performance gap fully. The gap makes it difficult to choose between direct numerical and LUT calculations. For this problem, a register-LUTs method with the nearest neighbor was proposed; however, it is limited for functions with narrow-range values approaching zero. In this paper, we propose a method for using register LUTs to process images efficiently over a wide range of values. Our contributions include proposing register-LUT with linear interpolation for efficient computation, using a smaller data type for further efficiency, and suggesting an efficient data retrieving method.

Abstract The resolution of cameras is increasing, and speedup of various image processing is required to accompany this increase. A simple way of acceleration is processing the image at low resolution and then upsampling the result. Moreover, when we can use an additional high-resolution image as guidance formation for upsampling, we can upsample the image processing results more accurately. We propose an approach to accelerate various image processing by downsampling and joint upsampling. This paper utilizes per-pixel look-up tables (LUTs), named local LUT, which are given a low-resolution input image and output pair. Subsequently, we upsample the local LUT. We can then generate a high-resolution image only by referring to its local LUT. In our experimental results, we evaluated the proposed method on several image processing filters and applications: iterative bilateral filtering, $$\ell _0$$ ℓ 0 smoothing, local Laplacian filtering, inpainting, and haze removing. The proposed method accelerates image processing with sufficient approximation accuracy, and the proposed outperforms the conventional approaches in the trade-off between accuracy and efficiency. Our code is available at https://fukushimalab.github.io/LLF/ .

Image compression is essential in image processing, and image quality assessment (IQA) is important in determining the image compression level. This study aims to construct a dataset for evaluating image quality at low compression in this coding degradation, i.e., for high-quality images. Typical IQA databases are selected for general-purpose image degradation, not high-quality images. If one tries to evaluate high-quality images, multi-level evaluations are difficult to construct successfully. In addition, the evaluated encoder is not a de facto standard encoding algorithm. Therefore, this study constructs a dataset for subjective evaluation of visually near-lossless level image compression quality based on binary level evaluation of the just noticeable difference (JND). Experimental results showed that the new dataset was validated for correlation by various IQAs. It was also shown that more than the compression quality covered by the conventional dataset is needed for the binary evaluation of high-quality images. The dataset is available at: https://norishigefukushima.github.io/iqanearlossless/.

We propose a novel blood volume pulse (BVP) signal extraction method for heart rate estimation that incorporates the self-similarity properties of BVP in the spatial and temporal domains. The main novelty of the proposed method is the incorporation of the temporal self-similarity of BVP via low-rank approximation in the time-delay coordinate system for BVP signal extraction. To make a low-rank approximation of BVP in the time domain, we introduce knowledge of linear time-invariant systems, i.e., the autoregressive (AR) model lies in the low-rank subspace in the time-delay coordinate system. In the medical field, it is widely known that BVP has quasi-periodic temporal characteristics owing to the cardiac pulse and exhibits self-similarity properties in the temporal domain. Hence, we model the temporal behavior of BVP as an AR process, allowing for a low-rank approximation of BVP in the time-delay coordinate system. Low-rank approximation of BVP in the time and spatial domains enables reliable BVP signal extraction, resulting in accurate heart rate estimation. The experiments demonstrate the effectiveness of the proposed method.

Multi-scale processing is essential in image processing and computer graphics. Halos are a central issue in multi-scale processing. Several edge-preserving decompositions resolve halos, e.g., local Laplacian filtering (LLF), by extending the Laplacian pyramid to have an edge-preserving property. Its processing is costly; thus, an approximated acceleration of fast LLF was proposed to linearly interpolate multiple Laplacian pyramids. This paper further improves the accuracy by Fourier series expansion, named Fourier LLF. Our results showed that Fourier LLF has a higher accuracy for the same number of pyramids. Moreover, Fourier LLF exhibits parameter-adaptive property for content-adaptive filtering.

This paper presents an autonomous parameter design method for a cascade structure feedback (FB) controller in industrial precision servo systems with resonance modes, considering a time and frequency domain optimization. In conventional autonomous design methods, parameters of a cascade structure FB controller are optimized by solving a frequency domain optimization problem to expand the control bandwidth and satisfy the desired stability margins. Therefore, a desired positioning response that is generally defined in time domain is not necessarily realized by the designed FB controller. The proposed method combines a time domain optimization problem to a conventional frequency domain optimization problem for improving the time domain positioning response. The effectiveness of the proposed method is demonstrated through an example FB controller design for a galvanometer scanner, in comparison with the conventional autonomous design method based on only the frequency domain parameter optimization.

Gaussian filtering is a smoothing filter used in various tasks. The main disadvantage is the dependence of the processing time on its kernel radius. One solution is using a sliding-discreet cosine transform (DCT), a constant-time algorithm for the kernel radius, and it provides the best performance in terms of both speed and accuracy. However, the speed and accuracy differ according to the type of DCT used. We can also accelerate the sliding-DCT based Gaussian filter by hardware accelerators, but the acceleration requires modification of the algorithms. In this paper, we focus on the fused multiply-add (FMA) instruction of hardware accelerators in modern computer architectures. The FMA instruction simultaneously performs multiplication and addition, i.e.,ax+b. We proposed an acceleration method of the sliding-DCT based Gaussian filtering for the FMA instruction. Moreover, we evaluate the performance of it in terms of computational time and approximation accuracy.

In this paper, we propose an acceleration method of redundant DCT filtering for deblurring and denoising. Current CCD cameras have high-resolution images, and the resolution has been increasing. Even if pixels are in focus, the pixels have slight blurring due to diffraction and Bayer interpolation. Therefore, we focus deblurring for slight blurring on real-time performance. Traditional approaches have a fast computational performance for this purpose, but these methods do not contain denoising architecture. In this paper, we simultaneously perform deblurring and denoising on the redundant DCT domain for accelerating the process. Also, we show that a post-scaling DCT can accelerate the proposed filtering. Experimental results show that the proposed method is the fastest method and the accuracy is also high among the fast approaches.

Image filtering is fundamental in image processing. The acceleration is essential since image resolution has highly increased. For the acceleration, image subsampling is a general approach for any filtering. However, this approach has a drawback in accuracy due to image aliasing. Several papers moderate this problem by sub-sampling image or filtering kernel non-sequential sampling. In this paper, we improve the sampling combined with image and kernel subsampling. We also accelerate the work of non-sequential sampling by vector addressing of hardware acceleration. Experimental results show that the proposed method accelerate bilateral filtering and adaptive Gaussian filtering. Also, the proposed vector addressing accelerate both filters.

In high-performance industrial mechatronic servo systems, control parameter tuning is necessary for achieving the desired positioning response to satisfy target control specifications. Auto-tuning, in particular, is a promising technology for efficient fine-tuning and is gaining popularity in industrial servo systems. However, fine-tuning of feedback (FB) control parameters is generally difficult since an aggressive parameter search could lead to the deterioration of control stability. Herein, we present the concept of advanced auto-tuning method considering a stable parameter area and an efficient calculation method of a stable parameter area to address auto-tuning of an FB controller. In this paper, the proposed calculation method of a stable parameter area is extended to an FB controller, which comprises a PID controller and some resonant compensation filters, and the calculation time and accuracy are comparatively evaluated with a full search-based calculation method.

This paper presents an efficient hybrid optimization method to design a high-performance cascade structure feedback (FB) controller for a mechatronic system with resonant vibration modes. The proposed method combines sequential quadratic programming and a genetic algorithm (GA) in order to obtain the parameters of the cascade structure FB controller. The proposed method makes it possible to obtain a wide-bandwidth FB controller that satisfies the specified stability margins and sensitibity function gains for resonant modes in a short design time. This paper describes the hybrid optimization method and design procedure for the cascade structure FB controller in detail. In addition, it shows how the effectiveness of the proposed method was evaluated through frequency-domain simulations using a laboratory galvano scanner, in comparison with the conventional GA-based optimization method.

This paper presents a friction model-based frequency response analysis (FRA) method, which gives a precise linear mechanical dynamics model to design effective controllers and analyze accurate control characteristics for frictional servo systems. As well known, frequency-domain identification approaches using a sine sweep are widely used to obtain linear dynamics. However, nonlinear friction in the mechanism varies the apparent frequency-domain characteristic of the linear dynamics due to the nonlinearity. The proposed FRA estimates effective excitation thrust for actual linear dynamics in the sine sweep movement, by means of a friction model as well as a phase delay model. Theoretical analyses show that the proposed FRA can identify the correct linear dynamics, preventing influence of nonlinear friction as well as phase delay properties included in a plant system. The effectiveness of the proposed FRA is verified through theoretical analyses and experiments both in frequency and time domains, in comparison to two conventional FRA methods.

This paper presents an iterative learning control (ILC) design technique for the fast and precise positioning of galvano scanners with parameter fluctuations. ILC scheme is well-known as a strong control approach for high-precision motion control of mechatronic systems. However, since ILC is heavily sensitive to modeling errors and uncertainty, resonant frequency fluctuations in the galvano scanner greatly affect stability of ILC and deteriorate the learning control performance. In this study, therefore, an ILC approach using zero phase error notch filters is presented to improve the control performance with the ILC stability robustly satisfied. Effectiveness of the proposed ILC has been verified by numerical simulations using a galvano scanner.

This paper presents a novel proportional-integral-derivative (PID) controller design considering the robust stability against plant perturbations for the fast and precise positioning control of mechatronic systems. Since parameter fluctuations in plant mechanisms and/or actuators, due to temperature variations, aged deteriorations, etc., generally deteriorate the motion performance as well as the system stability, an improvement in disturbance suppression capability of feedback (FB) control system is a general and important index to provide robust properties against the fluctuations. In this study, therefore, a circle condition-based PID controller design considering the robust stability is presented as well as a genetic algorithm (GA)-based optimization process of a PID-type FB controller with resonance compensation filters, to balance a trade-off between the disturbance suppression and the system stability. Effectiveness of the proposed approach has been verified by numerical simulations using a laboratory prototype of galvano scanner.