Photon-counting computed tomography (PCCT) based on photon-counting detectors (PCDs) represents a cutting-edge CT technology, offering higher spatial resolution, reduced radiation dose, and advanced material decomposition capabilities. Accurately modeling complex and nonlinear PCDs under limited calibration data becomes one of the challenges hindering the widespread accessibility of PCCT. This paper introduces a physics-ASIC architecture-driven deep learning detector model for PCDs. This model adeptly captures the comprehensive response of the PCD, encompassing both sensor and ASIC responses. We present experimental results demonstrating the model's exceptional accuracy and robustness with limited calibration data. Key advancements include reduced calibration errors, reasonable physics-ASIC parameters estimation, and high-quality and high-accuracy material decomposition images.

Hyperspectral images (HSIs) have widespread applications in geoscience, environmental monitoring, and resource management. However, in practical engineering applications, random stripe noise in HSIs severely affects data quality and accuracy, impacting the subsequent use of HSIs. In this paper, we propose an HSI intensity adaptive destriping method based on reference image-guided pixel clustering. The proposed method removes stripe noise through three main stages. First, the HSI is analyzed to manually select a specific band image with minimal impact from random stripe noise. This image is designated the initial reference image, and its specific location is identified. Second, the proposed pixel threshold classification method, deep maximum inter-class variance, is used for pixel threshold classification for band images that are contaminated with random stripe noise and adjacent to the reference image. Finally, the resulting pixels are classified into two categories according to noise intensity: high intensity and low intensity. The proposed intensity adaptive grayscale value reconstruction algorithm is used to remove stripe noise from each pixel category, and the denoised image is updated as a new reference image. Starting from the initial reference image, these steps are repeated along the spectrum to achieve destriping of all bands. We compare the proposed method against traditional and deep learning methods using real HSIs. The experimental results show that after denoising of the image using our method, both visual quality and quantitative evaluation metrics are significantly improved, with particularly excellent stripe removal performance observed for HSIs with significant grayscale variation.

Remote sensing images are increasingly being used in military and civilian fields. However, due to the influence of factors such as detector movement and temperature changes during the acquisition of remote sensing images, these images are often contaminated by stripe noise, which has adverse effects on tasks such as inversion, target detection, and semantic segmentation. Therefore, we developed a method for completely removing stripe noise, without destroying necessary information in remote sensing images, based on global search optimization and dense Gabor filters. Unlike other algorithms that directly solve the underlying image, this method applies Gabor filters to estimate stripe noise in the noisy image, locate stripe or nonstripe noise, and then categorize the stripe noise as sparse or dense according to its distribution frequency. Finally, the directionality of the stripe noise, its smoothness along the noise direction, and the local continuity of the underlying image are fully determined. The noisy image is divided into blocks along the stripe noise direction, and the intensity range of the stripe noise is estimated at different positions in each subimage. The stripe noise is directly solved within the estimated intensity range using global search optimization to achieve stripe noise removal. The proposed method can select different solutions for processing according to the type of stripe noise. A large number of experiments were conducted using simulated and real data, and the results demonstrated that the proposed method qualitatively and quantitatively outperformed current state-of-the-art stripe noise removal methods.

Achieving color consistency is essential for stitching large-area optical satellite imagery. The narrow swath width of individual images, combined with varying acquisition conditions, inherently introduces color differences. These manifest as marked disparities in brightness, color tone, and local contrast, degrading overall regional consistency. Existing methods primarily focus on correcting color inconsistencies between adjacent images, while often overlooking intra-image illumination non-uniformity, thereby propagating radiometric errors into optimization frameworks. Whisk-broom sensors, which can acquire imagery over a much wider swath width along the parallels, frequently exhibit substantial intra-image brightness variations, particularly in high-latitude regions. Compounded by frequent cloud cover and rapid temporal changes of features, extracting reliable color correspondences for optimization becomes intractable. To address these challenges, we propose a novel color correction framework that simultaneously considers intra-image illumination non-uniformity and radiometric variations caused by cloud prevalence and dynamic surface changes. First, a solar elevation angle map is extracted for down-sampled source image based on their geographic metadata and sensor geometry. An inverse compensation based on the normalized sine value of the solar elevation angle is then applied to mitigate brightness disparities caused by varying incident radiance. Second, to address atmospheric effects that vary with wavelength, such as differential absorption and scattering that cause color casts especially in low-illumination regions, a reference spectral channel is selected to guide the correction. Finally, we introduce a hybrid strategy for selecting reliable color correspondences in overlapping regions, using both grayscale and texture similarity under complex coverage conditions. Residual radiometric information is incorporated into a cost function, which jointly considers original color control and overall color balance to enhance the global consistency of the corrected mosaic. Extensive experiments conducted on imagery from the Wide Swath Imager (WSI) of DaQi-1 (DQ-1) and the Chinese Ocean Color and Temperature Scanner (COCTS) of HaiYang-1E (HY-1E) demonstrate that the proposed method effectively removes uneven illumination and color discrepancies. Compared to three state-of-the-art methods, our approach achieves superior performance in both visual quality and quantitative metrics.

The aim of end-to-end sign language translation (SLT) is to interpret continuous sign language (SL) video sequences into coherent natural language sentences without any intermediary annotations, i.e., glosses. However, end-to-end SLT suffers several intractable issues: (i) the temporal correspondence constraint loss problem between SL videos and glosses, and (ii) the weakly supervised sequence labeling problem between long SL videos and sentences. To address these issues, we propose an adaptive video representation enhanced Transformer (AVRET), with three extra modules: adaptive masking (AM), local clip self-attention (LCSA) and adaptive fusion (AF). Specifically, we utilize the first AM module to generate a special mask that adaptively drops out temporally important SL video frame representations to enhance the SL video features. Then, we pass the masked video feature to the Transformer encoder consisting of LCSA and masked self-attention to learn clip-level and continuous video-level feature information. Finally, the output feature of encoder is fused with the temporal feature of AM module via the AF module and use the second AM module to generate more robust feature representations. Besides, we add weakly supervised loss terms to constrain these two AM modules. To promote the Chinese SLT research, we further construct CSL-FocusOn, a Chinese continuous SLT dataset, and share its collection method. It involves many common scenarios, and provides SL sentence annotations and multi-cue images of signers. Our experiments on the CSL-FocusOn, PHOENIX14T, and CSL-Daily datasets show that the proposed method achieves the competitive performance on the end-to-end SLT task without using glosses in training. The code is available at https://github.com/LzDddd/AVRET.

The Segment Anything Model (SAM) is a foundational model that has demonstrated impressive results in the field of natural image segmentation. However, its performance remains suboptimal for medical image segmentation, particularly when delineating lesions with irregular shapes and low contrast. This can be attributed to the significant domain gap between medical images and natural images on which SAM was originally trained. In this paper, we propose an adaptation of SAM specifically tailored for lesion segmentation termed LeSAM. LeSAM first learns medical-specific domain knowledge through an efficient adaptation module and integrates it with the general knowledge obtained from the pre-trained SAM. Subsequently, we leverage this merged knowledge to generate lesion masks using a modified mask decoder implemented as a lightweight U-shaped network design. This modification enables better delineation of lesion boundaries while facilitating ease of training. We conduct comprehensive experiments on various lesion segmentation tasks involving different image modalities such as CT scans, MRI scans, ultrasound images, dermoscopic images, and endoscopic images. Our proposed method achieves superior performance compared to previous state-of-the-art methods in 8 out of 12 lesion segmentation tasks while achieving competitive performance in the remaining 4 datasets. Additionally, ablation studies are conducted to validate the effectiveness of our proposed adaptation modules and modified decoder.

Low-dose computed tomography (LDCT) imaging faces great challenges. Although supervised learning has revealed great potential, it requires sufficient and high-quality references for network training. Therefore, existing deep learning methods have been sparingly applied in clinical practice. To this end, this paper presents a novel Unsharp Structure Guided Filtering (USGF) method, which can reconstruct high-quality CT images directly from low-dose projections without clean references. Specifically, we first employ low-pass filters to estimate the structure priors from the input LDCT images. Then, inspired by classical structure transfer techniques, deep convolutional networks are adopted to implement our imaging method which combines guided filtering and structure transfer. Finally, the structure priors serve as the guidance images to alleviate over-smoothing, as they can transfer specific structural characteristics to the generated images. Furthermore, we incorporate traditional FBP algorithms into self-supervised training to enable the transformation of projection domain data to the image domain. Extensive comparisons and analyses on three datasets demonstrate that the proposed USGF has achieved superior performance in terms of noise suppression and edge preservation, and could have a significant impact on LDCT imaging in the future.

Medical image denoising faces great challenges. Although deep learning methods have shown great potential, their efficiency is severely affected by millions of trainable parameters. The non-linearity of neural networks also makes them difficult to be understood. Therefore, existing deep learning methods have been sparingly applied to clinical tasks. To this end, we integrate known filtering operators into deep learning and propose a novel Masked Joint Bilateral Filtering (MJBF) via deep image prior for digital X-ray image denoising. Specifically, MJBF consists of a deep image prior generator and an iterative filtering block. The deep image prior generator produces plentiful image priors by a multi-scale fusion network. The generated image priors serve as the guidance for the iterative filtering block, which is utilized for the actual edge-preserving denoising. The iterative filtering block contains three trainable Joint Bilateral Filters (JBFs), each with only 18 trainable parameters. Moreover, a masking strategy is introduced to reduce redundancy and improve the understanding of the proposed network. Experimental results on the ChestX-ray14 dataset and real data show that the proposed MJBF has achieved superior performance in terms of noise suppression and edge preservation. Tests on the portability of the proposed method demonstrate that this denoising modality is simple yet effective, and could have a clinical impact on medical imaging in the future.