In stereoscope-based minimally invasive surgeries (MISs), dense stereo matching plays an indispensable role in 3-D shape recovery, AR, VR, and navigation tasks. Although numerous deep neural network (DNN) approaches are proposed, the conventional prior-free approaches are still popular in the industry because of the lack of open-source annotated dataset and the limitation of the task-specific pretrained DNNs. Among the prior-free stereo matching algorithms, there is no successful real-time algorithm in none GPU environment for MIS. This article proposes the first CPU-level real-time prior-free stereo matching algorithm for general MIS tasks. We achieve an average$14-17$Hz on$640 \times 480$images with a single-core CPU (i5-9400) for surgical images. Meanwhile, it achieves slightly better accuracy than the popular efficient large-scale stereo matching (ELAS) method. The patch-based fast disparity searching algorithm is adopted for the rectified stereo images. A coarse-to-fine Bayesian probability and a spatial Gaussian mixed model were proposed to evaluate the patch probability at different scales. An optional probability density function estimation algorithm was adopted to quantify the prediction variance. Extensive experiments demonstrated the proposed method's capability to handle ambiguities introduced by the textureless surfaces and the photometric inconsistency from the non-Lambertian reflectance and dark illumination. The estimated probability managed to balance the confidences of the patches for stereo images at different scales. It has similar or higher accuracy and fewer outliers than the baseline ELAS in MIS, while it is 4–5 times faster.

Digital images from today's premium digital cameras have a higher dynamic range than what humans can perceive. Maximumly keeping the image information captured by sensors while reducing the file size of the output raw image data is highly desirable. In this paper, we show that the extractable image information contained in high-quality HDR images taken by premium cameras can be preserved using a bit-depth of only 8 to 9 bits/pixel when the shot-noise-introduced extra entropy is suppressed. The quadratic variable step-size quantization is used to suppress the shot-noise-introduced entropy. The ratio of the quantization step-size of the quadratic quantizer over the shot noise amplitude is introduced as a parameter to measure the relative quantization noise level, and is used to design the quadratic quantizers that maximally suppress the shot noise. Finally, quadratically quantized real CFA camera-raw image data are losslessly compressed. Compared with the lossless compression on linearly quantized data, at least more than 50% bitrate savings are achieved. Such high bitrate savings without loss of extractable image information are very attractive. Thus, it is highly worthwhile to explore, in the future, hardware implementations of quadratic quantization in image sensors.

A new video coding algorithm using 3D transform is introduced. The algorithm does not use motion compensation. Two novel parts of the algorithm are: (1) The 4-channel SCWP is used for the time-dimension transform, and (2) A novel run-length coding method is developed for coding the quantized transform coefficients. Principally, the complexity of the proposed transform video coding algorithm is of the same order of magnitude to that of a 2D still image transform coding algorithm. However, its compression performance is competitive to HEVC at high compression bitrates.

Tissue surface shape and reflectance spectra provide rich intra-operative information useful in surgical guidance. We propose a hybrid system which displays an endoscopic image with a fast joint inspection of tissue surface shape using structured light (SL) and hyperspectral imaging (HSI). For SL a miniature fibre probe is used to project a coloured spot pattern onto the tissue surface. In HSI mode standard endoscopic illumination is used, with the fibre probe collecting reflected light and encoding the spatial information into a linear format that can be imaged onto the slit of a spectrograph. Correspondence between the arrangement of fibres at the distal and proximal ends of the bundle was found using spectral encoding. Then during pattern decoding, a fully convolutional network (FCN) was used for spot detection, followed by a matching propagation algorithm for spot identification. This method enabled fast reconstruction (12 frames per second) using a GPU. The hyperspectral image was combined with the white light image and the reconstructed surface, showing the spectral information of different areas. Validation of this system using phantom and ex vivo experiments has been demonstrated.

In multi-pinhole SPECT, overlapping the projections from different pinholes has been used to increase sensitivity. However, the prevailing view is that the overall quality of the reconstructed image is not improved by the overlaps in the projections. It is often stated in literatures that overlaps introduce ambiguous information, which can lead to nonuniqueness of solution for the inverse problem, and thus artifacts are introduced in the reconstructed image. On the other hand, contrary to the prevailing view, a recent study on slit-slat collimators shows that artifacts can be removed with the "help" of an extra complete nonoverlapped projection data set. In this paper, two types of artifact-free projection overlaps are defined in general, and the criteria for designing artifact-free multi-pinhole systems with overlaps are proposed. It is shown that once the criteria are satisfied, the solution of the inverse problem is unique, and thus no artifact is expected in the reconstructed image. Via extensive simulation study, various artifact-free overlapping multi-pinhole systems are designed and validated. It is shown that overlaps in the artifact-free systems can improve contrast-to-noise ratio (CNR). With a proper design, the CNR for an artifact-free overlapping system can be significantly higher than that for the corresponding nonoverlapping system. The improved image quality is also confirmed with noisy reconstructions.

Two-band analysis-synthesis filters or wavelet filters are used pervasively for compressing natural images. Both FIR and IIR filters have been studied in this context, the former being the most popular. In this paper, we examine the compression performance of these two-band filters in a dyadic wavelet decomposition and attempt to isolate features that contribute most directly to the performance gain. Then, employing the general exact reconstruction condition, hybrid FIR-IIR analysis-synthesis filters are designed to maximize compression performance for natural images. Experimental results are presented that compare performance with the popular biorthogonal filters in terms of peak SNR, subjective quality, and computational complexity.

In this correspondence, a new two-channel overlapped block transform is introduced by prefiltering the Haar transform. The prefiltering employs plane rotations with one single rotation angle to achieve high computational efficiency. The resulting transform is equivalent to a periodically time-varying (PTV) filter bank. The proposed transform or PTV filter bank has several advantages for image coding. First, symmetric extension is very easily performed at the boundaries in spite of the fact that the filters are asymmetric. Second, arithmetic complexity can be reduced to simple shifts and adds. Finally, the performance for image compression is shown to be high, both in terms of visual quality and PSNR. As a result, the proposed transform is a competitive alternative to the 5/3 and the 9/7 filters, which are currently popular for subband/wavelet image coders.

Subband decompositions for image coding have been explored extensively over the last few decades. The condensed wavelet packet (CWP) transform is one such decomposition that was recently shown to have coding performance advantages over conventional decompositions. A special feature of the CWP is that its design and implementation are performed in the cyclic frequency domain. While performance gains have been reported, efficient implementations of the CWP (or more generally, efficient implementations of cyclic filter banks) have not yet been fully explored. In this paper, we present efficient block-based implementations of cyclic filter banks along with an analysis of the arithmetic complexity. Block-based cyclic filter bank implementations of the CWP coder are compared with conventional subband/wavelet image coders whose filter banks are implemented in the time domain. It is shown that block-based cyclic filter bank implementations can result in CWP coding systems that outperform the popular image coding systems both in terms of arithmetic complexity and coding performance.

In subband/wavelet image coding, size-limited subband decompositions are ordinarily used to avoid increasing the number of samples that need to be coded. To reduce coding distortions that can occur at the borders, the symmetric extension filter bank is typically employed. This paper introduces some new perspectives and improvements to that decomposition. The symmetric extension filter bank is couched in the cyclic frequency domain, providing a framework that accommodates FIR and IIR filters in a natural way, all with perfect reconstruction. IIR filters with both rational and irrational transfer functions can be implemented and, in the context of symmetric extension, can accommodate IIRs that effectively have perfect stopband suppression. Enhancements to the filter bank at a tree-structured system level are also presented and include the application of spectral reversal correction and a transition band normalization approach to designing the constituent filters of the symmetric extension wavelet packet transform.

Symmetric extension is typically employed in subband/wavelet image coders to improve compression performance, particularly at the image boundaries. This paper introduces improvements to the symmetric extension filter bank. The filter bank implementation employs a cyclic frequency domain representation and is able to accommodate IIR approximations that effectively have perfect stopband suppression. Enhancements are also introduced at the multi-rate system level. The new implementation offers greater flexibility in choice of filters and is shown to result in better compression performance.