Conventional radiography is the widely used imaging technology in diagnosing, monitoring, and prognosticating musculoskeletal (MSK) diseases because of its easy availability, versatility, and cost-effectiveness. Bone overlaps are prevalent in conventional radiographs, and can impede the accurate assessment of bone characteristics by radiologists or algorithms, posing significant challenges to conventional clinical diagnosis and computer-aided diagnosis. This work initiated the study of a challenging scenario - bone layer separation in conventional radiographs, in which separate overlapped bone regions enable the independent assessment of the bone characteristics of each bone layer and lay the groundwork for MSK disease diagnosis and its automation. This work proposed a Bone Layer Separation GAN (BLS-GAN) framework that can produce high-quality bone layer images with reasonable bone characteristics and texture. This framework introduced a reconstructor based on conventional radiography imaging principles, which achieved efficient reconstruction and mitigates the recurrent calculations and training instability issues caused by soft tissue in the overlapped regions. Additionally, pre-training with synthetic images was implemented to enhance the stability of both the training process and the results. The generated images passed the visual Turing test, and improved performance in downstream tasks. This work affirms the feasibility of extracting bone layer images from conventional radiographs, which holds promise for leveraging layer separation technology to facilitate more comprehensive analytical research in MSK diagnosis, monitoring, and prognosis.

Accurate segmentation of wrist bones from conventional radiographs remains a significant challenge due to severe anatomical overlap and blurred bone boundaries, particularly in patients with rheumatoid arthritis. To address these issues, we propose AP-DPM, a novel Dual-Path Prior-constrained Merging model with adversarial anatomical priors. AP-DPM employs a dual-path architecture to separately predict complete bone masks and overlapping regions, which are subsequently integrated through a residual merging network. To enhance anatomical plausibility, we incorporate an adversarial prior guided by a pre-trained discriminator. Extensive experiments on the publicly available RAM-W600 dataset demonstrate that AP-DPM outperforms state-of-the-art methods across seven quantitative metrics, achieving superior performance particularly in diagnostically critical overlapping regions. Ablation studies further validate that both the dualpath structure for enhanced focusing on overlap regions and the adversarial anatomical prior contribute significantly to performance gains, enhancing local boundary sensitivity and global structural consistency. These results highlight the potential of AP-DPM to improve automated radiographic assessment of rheumatoid arthritis progression. Code is available at https://github.com/YSongxiao/AP-DPM

Rheumatoid arthritis (RA) is a chronic autoimmune disease characterized by joint inflammation and progressive structural damage. Joint space width (JSW) is a critical indicator in conventional radiography (CR) for evaluating disease progression, which has become a prominent research topic in computer-aided diagnostic (CAD) systems. However, deep learning-based radiological CAD systems for JSW analysis face significant challenges in data quality, including data imbalance, limited variety, and annotation difficulties. This work introduced a challenging image synthesis scenario and proposed Layer Separation Networks (LSN) to accurately separate the soft tissue layer, the upper bone layer, and the lower bone layer in conventional radiographs of finger joints. Using these layers, the adjustable JSW images can be synthesized to address data quality challenges and achieve ground truth (GT) generation. Experimental results demonstrated that LSN-based synthetic images closely resemble real radiographs, and significantly enhanced the performance in downstream tasks. The code and dataset are available at: https://github.com/pokeblow/LSN.

Rheumatoid arthritis (RA) is a common autoimmune disease that has been the focus of research in computer-aided diagnosis (CAD) and disease monitoring. In clinical settings, conventional radiography (CR) is widely used for the screening and evaluation of RA due to its low cost and accessibility. The wrist is a critical region for the diagnosis of RA. However, CAD research in this area remains limited, primarily due to the challenges in acquiring high-quality instance-level annotations. (i) The wrist comprises numerous small bones with narrow joint spaces, complex structures, and frequent overlaps, requiring detailed anatomical knowledge for accurate annotation. (ii) Disease progression in RA often leads to osteophyte, bone erosion (BE), and even bony ankylosis, which alter bone morphology and increase annotation difficulty, necessitating expertise in rheumatology.This work presents a multi-task dataset for wrist bone in CR, including two tasks: (i) wrist bone instance segmentation and (ii) Sharp/van der Heijde (SvdH) BE scoring, which is the first public resource for wrist bone instance segmentation. This dataset comprises 1048 wrist conventional radiographs of 388 patients from six medical centers, with pixel-level instance segmentation annotations for 618 images and SvdH BE scores for 800 images. This dataset can potentially support a wide range of research tasks related to RA, including joint space narrowing (JSN) progression quantification, BE detection, bone deformity evaluation, and osteophyte detection. It may also be applied to other wrist-related tasks, such as carpal bone fracture localization.We hope this dataset will significantly lower the barrier to research on wrist RA and accelerate progress in CAD research within the RA-related domain.Benchmark & Code: https://github.com/YSongxiao/RAM-W600Data & Dataset Card: https://huggingface.co/datasets/TokyoTechMagicYang/RAM-W600

Presents corrections to the article "A Sub-Pixel Accurate Quantification of Joint Space Narrowing Progression in Rheumatoid Arthritis".

Rheumatoid arthritis (RA) is a chronic autoimmune disease that primarily affects peripheral synovial joints, like fingers, wrists and feet. Radiology plays a critical role in the diagnosis and monitoring of RA. Limited by the current spatial resolution of radiographic imaging, joint space narrowing (JSN) progression of RA for the same reason above can be less than one pixel per year with universal spatial resolution. Insensitive monitoring of JSN can hinder the radiologist/rheumatologist from making a proper and timely clinical judgment. In this paper, we propose a novel and sensitive method that we call partial image phase-only correlation which aims to automatically quantify JSN progression in the early RA. The majority of the current literature utilizes the mean error, root-mean-square deviation and standard deviation to report the accuracy at pixel level. Our work measures JSN progression between a baseline and its follow-up finger joint images by using the phase spectrum in the frequency domain. Using this study, the mean error can be reduced to 0.0130 mm when applied to phantom radiographs with ground truth, and 0.0519 mm standard deviation for clinical radiography. With the sub-pixel accuracy far beyond usual manual measurements, we are optimistic that the proposed work is a promising scheme for automatically quantifying JSN progression.

The rising demand for high quality display has ensued active research in high dynamic range (HDR) imaging, which has the potential to replace the standard dynamic range imaging. This is due to HDR’s features like accurate reproducibility of a scene with its entire spectrum of visible lighting and color depth. But this capability comes with expensive capture, display, storage and distribution resource requirements. Also, display of HDR images/video content on an ordinary display device with limited dynamic range requires some form of adaptation. Many adaptation algorithms, widely known as tone mapping (TM) operators, have been studied and proposed in the last few decades. In this article, we present a comprehensive survey of 60 TM algorithms that have been implemented on hardware for acceleration and real-time performance. In this state-of-the-art survey, we will discuss those TM algorithms which have been implemented on GPU, FPGA, and ASIC in terms of their hardware specifications and performance. Output image quality is an important metric for TM algorithms. From our literature survey we found that, various objective quality metrics have been used to demonstrate the quality of those algorithms hardware implementation. We have compiled those metrics used in this survey, and analyzed the relationship between hardware cost, image quality and computational efficiency. Currently, machine learning-based (ML) algorithms have become an important tool to solve many image processing tasks, and this article concludes with a discussion on the future research directions to realize ML-based TM operators on hardware.

We present a fast global and locally adaptive tone mapping algorithm and its field-programmable gate array (FPGA) implementation. The specially designed tone mapping function, which is based on local histogram equalization, controls global, and local characteristics individually. In contrast to other tonemap operators, our algorithm manages light/dark halos separately and by using local tonemap function alone, it can effectively suppress noise. We validated the effectiveness of our algorithms using subjective and objective assessment. Using an average of the bins, we achieve fast smoothed local histogram estimation with fewer bins while maintaining high accuracy. Our new implementation method requires minimal data access and reduced memory as it operates with a downscaled frame size of 240 × 135 pixels. Relative local area size is 248 × 248 @Full-HD resolution (1920 × 1080). For low-latency pixel output, the system performs the tone mapping using pixel information from the previous frame. When we implemented the system on FPGA (TB-7K-325TIMG and Xilinx Kintex-7), we achieved lightweight hardware as the total usage rate is about 25% of the available FPGA resource. Using an online 1080p video we demonstrate, a real-time video processing using our hardware tone mapping system.

Energy-constrained neural network processing is in high demanded for various mobile applications. Binary neural network aggressively enhances the computational efficiency, and in contrast, it suffers from degradation of accuracy due to its extreme approximation. We propose a novel accurate neural network model based on binarization and "dithering" that distributes the quantization error to neighboring pixels. The quantization errors in the binarization are distributed in the plane, so that a pixel in the multi-level source expression more accurately represented in the resulting binarized plane by multiple pixels. We designed a low-overhead binary-based hardware architecture for the proposed model. The evaluation results show that this method can be realized with a few additional lightweight hardware components.

In this study, we propose an efficient stereo matching method which estimates sparse disparities using global phase only correlation (POC). Conventionally, cost functions are to be calculated for all disparity candidates and the associated computational cost has been impediment in achieving a realtime performance. Therefore, we consider to use full image 2D phase only correlation (FIPOC) for detecting the valid disparities candidates. This would require comparatively fewer calculations for the same number of disparities. Since, the FIPOC output indicates the disparity distribution of two stereo images, we can sort the disparity candidates and choose them for sparse calculation. In our proposed method, the searchable disparity range is half of the input image size, which is much wider than that of the conventional methods. When we apply the FIPOC to naive sum of absolute difference (SAD) stereo matching method, the combined algorithm would require fewer calculations while maintaining the same accuracy. In our evaluation, the proposed method achieves 194 disparity stereo matching in 70 ms on$398 \times 288$images without the need for SIMD instruction, multi-thread operation, or additional hardware while using a Intel Core i5-5257U.

To replicate human visual perception, we analyze processing images with optical illusion using edge preserving filters and smoothed local histogram equalization (LHE). Images with the optical illusions are good models for gradual/rapid changes in contrast and strong edges, which are good cases for assessing the robustness of image filters. Here, we study and analyze the performance of smoothed LHE filters while processing perceptual illusion. Our studies conclude that, smoothed LHEs are useful in retaining actual edge forms in these images as they can operate using large kernel sizes. These large kernel size filters can construct sawtooth like edge and it corresponds to adequately wide halos. We also demonstrate the usefulness of smoothed LHE like tone mapping techniques in preserving naturalness, and we confirmed it by performing subjective visual test.

A 1,000-fps motion vector (MV) estimation and classification engine for high-speed computational imaging in a 3D stacked imager/processor module is proposed, prototyped, assembled and tested. The module features 1) ThruChip interfaces for high fps image transfer, 2) orders of magnitude more area/power efficient MV estimation architecture compared to conventional ones, and 3) a cognitive classification scheme employed on MV patterns, enabling the classification of moving objects not possible in conventional proposals.

The widely accepted block-matching technique, which is required to identify motion vectors, fails in cases in which texture is not existent. In [1], we proposed a hardware-oriented cellular-automaton algorithm that generates spatial patterns on textureless objects and backgrounds, aiming at motion-vector estimation of textureless moving objects. This demonstration presents a field-programmable gate array (FPGA) system that supports real-time processing. This system provides motion-vectors in moving textureless objects and enables enhanced processing of motion vector classification.

The convolutional neural network (CNN) is a state-of-the-art model that can achieve significantly high accuracy in many machine-learning tasks. Recently, for further developing the practical applications of CNNs, efficient hardware platforms for accelerating CNN have been throughly studied. A binarized neural network has been reported to minimize the multipliers, which consume a large amount of resources, with a minimal decrease in accuracy. In this study, we analyzed the optimal performance of CNN implemented on an field programmable gate array (FPGA) considering its logic resources and a memory bandwidth, using multiple types of parallelisms such as kernels, pixels, and channels both in conventional and binarized CNNs. As a result, it became clear that all the parallelisms are required for the binarized neural network to obtain the best performance of 8.38 TOPS.

We propose a bit-depth expansion (BDE) method targeting natural images. In the analog part of an imaging system, signal intensity fluctuations occur due to noise (e.g. thermal noise in the image sensor). After that, in the digital part, the intensities are rounded off to limited levels. The latter process, which is quantization, increases the intensity of fluctuation errors caused by stochastic resonance. These errors are viewed as false contour artifacts in the gradation region. Our goal was to obtain the original signal from the quantized noisy signal. We formulated a probabilistic model based on this quantization process, and successfully reconstructed smooth gradations from noisy contours. Subjective evaluation by voting clarified that the output image has higher quality.

We propose a halo control method that is suitable for smoothed local histogram equalization (LHE). Smoothed LHE based tone mapping, e.g., Apical Iridix®, is effective and widely used for local adaptive image enhancement. However, halo artifacts occur at the edge, which causes unnaturalness. There are no detailed discussions on the principles of smoothed LHE halo-artifacts. In this paper, we analyze the cause of the artifacts and propose a halo reduction/control method that uses a weighted local histogram. In addition, we separate the artifacts into light and dark and evaluate the effectivity for the human visual sense. The adequate weighting function of our method can be set automatically at each pixel by using the difference between the input and mean luminance of the local window so as to enhance the natural local edge.

We propose a method for improving the accuracy of histogram-based image filtering. With this method, we define a histogram called intensity-stacked histogram. An image histogram generally consists of a frequency (number of pixels) for each bin. On the other hand, intensity-stacked histogram stores the sum of intensity values for each bin. The intensity-stacked histogram can be calculated in constant time similar to a standard histogram. We apply the intensity-stacked histogram to histogram-based image algorithms for median and bilateral filters. The histogram-based image filter with the intensity-stacked histogram works effectively when using a few bins. We confirmed that the accuracy of histogram-based image filters using intensity-stacked histogram is higher than that using standard histogram.

We propose a O(1) algorithm for bilateral filter with low memory usage. Bilateral filter can be converted into weighted histogram operation. Applying line buffers of column histograms, we can reduce the number of calculation needed to construct recursive center-weighted local histogram. Also our method have advantage in terms of memory requirements. We used a 2-GHz CPU with our method and achieved one million pixels per 0.5 sec operation and high PSNR over 40 dB without the need for temporary frame buffers or additional instructions (downsampling, SIMD instructions, or multi-thread operations).

We describe a high-speed method of correcting and compressing the dynamic range of images that can be operated intuitively and naturally. Adaptive operations are conducted for shadow, middle, and highlight tones in the local areas of images. Although natural image processing can be achieved with default parameters, we can set the parameters for each of these three tones individually and intuitively. We attained two million pixels per 0.3 sec of operation and real-time VGA movie processing without the need for SIMD instruction, multi-thread operation, or additional hardware by using the Athlon64 X2 3800+ CPU.

This paper proposes circuit construction for cellular-automaton devices that can perform morphological picture processing. To produce the complex cell functions required for the morphological processing, we propose using a silicon functional MOS device, the /spl nu/-MOS FET. We designed sample cell circuits for several morphological processings, and simulated their operation. We also designed a sample cellular-automaton circuit using the proposed cell circuits, and demonstrated in simulation that the /spl nu/-MOS cellular-automata circuits are very useful for constructing intelligent image sensors.