Self-supervised monocular depth estimation serves as a key task in the development of endoscopic navigation systems. However, performance degradation persists due to uneven illumination inherent in endoscopic images, particularly in low-intensity regions. Existing low-light enhancement techniques fail to effectively guide the depth network. Furthermore, solutions from other fields, like autonomous driving, require well-lit images, making them unsuitable and increasing data collection burdens. To this end, we present DeLightMono - a novel self-supervised monocular depth estimation framework with illumination decoupling. Specifically, endoscopic images are represented by a designed illumination-reflectance-depth model, and are decomposed with auxiliary networks. Moreover, a self-supervised joint-optimizing framework with novel losses leveraging the decoupled components is proposed to mitigate the effects of uneven illumination on depth estimation. The effectiveness of the proposed methods was rigorously verified through extensive comparisons and an ablation study performed on two public datasets.

Parkinson’s disease (PD) is a prevalent neurodegenerative disorder globally. The eye’s retina is an extension of the brain, and clinical evidence has suggested the great potential of retinal pathology as surrogate biomarkers for early PD diagnosis. In particular, recent studies have shown that texture features extracted from retinal layers based on optical coherence tomography (OCT) images are strongly associated with PD-related retinal pathology. Additionally, frequency domain learning techniques can improve the representational capabilities of deep neural networks (DNNs) by decomposing frequency components that involve rich texture features, which remain underexplored for automated early PD diagnosis in OCT. To bridge this gap, we propose an Adaptive Wavelet Filter (AWF) that serves as the Practical Texture Amplifier, which fully leverages the merits of texture features from the retinal pathology view. Specifically, AWF first enhances feature map diversities and refines feature representations via channel mixer, then emphasizes informative texture feature representations with the well-designed adaptive wavelet filtering token mixer with the aid of frequency domain learning. By embedding AWFs into the DNN stem, AWFNet is constructed for automated early PD screening from OCT images. Additionally, we introduce a novel Balanced Confidence (BC) loss to boost early PD screening performance and trustworthiness of AWFNet, by mining the potential of sample-wise predicted probabilities across all classes and class frequency prior. The extensive experiments manifest the superiority of AWFNet with BC over state-of-the-art methods in terms of early PD screening performance and trustworthiness.

Quantitative analysis of retinal vascular morphology is vital for clinical decision-making and the investigation of systemic diseases. Central to this process is the accurate segmentation of retinal arteries and veins (A/V) from the background, a task challenged by substantial variations in vessel calibers and the presence of low-contrast or ambiguous structures in fundus images, especially in ultra-wide field imaging where peripheral distortions and large-scale anatomical variability are pronounced. These factors often lead to fragmented semantic representations and topological inconsistencies in automated segmentation outputs. To address these limitations, we propose Ultra, a multi-granularity topological reasoning network designed for precise A/V segmentation. Ultra adopts a cascaded two-stage architecture: PriorNet generates coarse, multi-scale vascular priors that provide structural guidance, while RefineNet performs topology-aware segmentation refinement. To further enforce topological coherence, we propose the neighboring pixel connectivity regularization (NICER) layer, which selectively integrates local connectivity information predicted by the proposed connectivity prediction union (CPU) module. This connectivity is employed as auxiliary supervision through a pixel-wise local connectivity loss, reinforcing structural reasoning and promoting anatomically consistent vascular topology inference. Extensive experiments on ultra-wide field fundus imaging (UWF) datasets demonstrate that Ultra achieves state-of-the-art performance in A/V segmentation and topological preservation. Moreover, Ultra generalizes well to conventional color fundus photography (CFP) datasets, underscoring its robustness and broad applicability. Code is publicly available at: https://github.com/iMED-Lab/Ultra.

The robust segmentation of different targets in multiple modality images is challenging due to factors such as low contrast, variations in target size and shape, and interference from diseases, which may lead to segmentation ambiguity. In addition, the assessment of the reliability of artificial intelligence is crucial for its clinical application. This paper proposes the Online Bayesian approximation based Uncertainty-aware Network (OBU-Net) for robust ophthalmic image segmentation. Our approach introduces an efficient online Bayesian method to update a spatial uncertainty map during training continuously. Then, the Spatial Uncertainty Aware Block (SUA-B) leverages the uncertainty map to localize and prioritize attention to ambiguous regions. Additionally, we extract pixel-wise confidence from multi-scale predictions to integrate hierarchical predictions. We compare OBU-Net with state-of-the-art (SOTA) methods on six datasets. The experimental results demonstrate that our method achieves the best overall performance across different modalities and segmentation tasks, highlighting the robustness of our approach. Additionally, metamorphic testing experiments were conducted, exploring the algorithm's stability against random perturbations. Lastly, we propose an image-level uncertainty score and demonstrate its effectiveness for evaluating the model's segmentation reliability.

Efficient convolutional neural network (CNN) architecture design has attracted growing research interests. However, they typically apply single receptive field (RF), small asymmetric RFs, or pyramid RFs to learn different feature representations, still encountering two significant challenges in medical image classification tasks: i) They have limitations in capturing diverse lesion characteristics efficiently, e.g., tiny, coordination, small and salient, which have unique roles on the classification results, especially imbalanced medical image classification. ii) The predictions generated by those CNNs are often unfair/biased, bringing a high risk when employing them to real-world medical diagnosis conditions. To tackle these issues, we develop a new concept, Expert-Like Reparameterization of Heterogeneous Pyramid Receptive Fields (ERoHPRF), to simultaneously boost medical image classification performance and fairness. This concept aims to mimic the multi-expert consultation mode by applying the well-designed heterogeneous pyramid RF bag to capture lesion characteristics with varying significances effectively via convolution operations with multiple heterogeneous kernel sizes. Additionally, ERoHPRF introduces an expert-like structural reparameterization technique to merge its parameters with the two-stage strategy, ensuring competitive computation cost and inference speed through comparisons to a single RF. To manifest the effectiveness and generalization ability of ERoHPRF, we incorporate it into mainstream efficient CNN architectures. The extensive experiments show that our proposed ERoHPRF maintains a better trade-off than state-of-the-art methods in terms of medical image classification, fairness, and computation overhead. The code of this paper is available at https://github.com/XiaoLing12138/Expert-Like-Reparameterization-of-Heterogeneous-Pyramid-Receptive-Fields.

Decoupling domain-variant information (DVI) from domain-invariant information (DII) serves as a prominent strategy for mitigating domain shifts in the practical implementation of deep learning algorithms. However, in medical settings, concerns surrounding data collection and privacy often restrict access to both training and test data, hindering the empirical decoupling of information by existing methods. To tackle this issue, we propose an Adaptive Information Filter-driven Source-free Domain Adaptation (AIF-SFDA) algorithm, which leverages a frequency-based learnable information filter to autonomously decouple DVI and DII. Information Bottleneck (IB) and Self-supervision (SS) are incorporated to optimize the learnable frequency filter. The IB governs the information flow within the filter to diminish redundant DVI, while SS preserves DII in alignment with the specific task and image modality. Thus, the adaptive information filter can overcome domain shifts relying solely on target data. A series of experiments covering various medical image modalities and segmentation tasks were conducted to demonstrate the benefits of AIF-SFDA through comparisons with leading algorithms and ablation studies.

Large language models (LLMs) have demonstrated remarkable capabilities in various domains, including radiology report generation.Previous approaches have attempted to utilize multimodal LLMs for this task, enhancing their performance through the integration of domainspecific knowledge retrieval.However, these approaches often overlook the knowledge already embedded within the LLMs, leading to redundant information integration.To address this limitation, we propose RADAR, a framework for enhancing radiology report generation with supplementary knowledge injection.RADAR improves report generation by systematically leveraging both the internal knowledge of an LLM and externally retrieved information.Specifically, it first extracts the model's acquired knowledge that aligns with expert imagebased classification outputs.It then retrieves relevant supplementary knowledge to further enrich this information.Finally, by aggregating both sources, RADAR generates more accurate and informative radiology reports.Extensive experiments on MIMIC-CXR, CHEXPERT-PLUS, and IU X-RAY demonstrate that our model outperforms state-of-the-art LLMs in both language quality and clinical accuracy 1 .

Iris motion tracking is critical for discriminating the iris stiffness and developmental stage of primary angle-closure disease (PACD). Anterior segment optical coherence tomography (AS-OCT) video is a highly efficient approach to observe the morphological determinant in iris motion. However, the iris exhibits inconsistent elastic changes during movement, accompanied by changes in local features after long-term frames. Currently, iris tracking methods have not yet been studied in AS-OCT videos. In this paper, we propose a Temporal Constraint-based Tracking Morph (TCTMorph) for estimating iris trajectory in long-term AS-OCT videos. We first estimate the deformation fields between three interrelated frames by a multi-frame diffeomorphic registration network. Then, we estimate iris trajectory from these results in long-term AS-OCT video sequences by leveraging temporal constraints among the consecutive flows. Our experiments on multi-center AS-OCT glaucoma datasets demonstrate that our method outperforms conventional motion tracking methods for long-term iris trajectory tracking.

A comprehensive and explicit understanding of surgical scenes plays a vital role in developing context-aware computer-assisted systems in the operating theatre. However, few works provide systematical analysis to enable hierarchical surgical scene understanding. In this work, we propose to represent the tasks set [phase recognition$\rightarrow $step recognition$\rightarrow $action and instrument detection] as multi-level semantic scene understanding (MSSU). For this target, we propose a novel hierarchical context transformer (HCT) network and thoroughly explore the relations across the different level tasks. Specifically, a hierarchical relation aggregation module (HRAM) is designed to concurrently relate entries inside multi-level interaction information and then augment task-specific features. To further boost the representation learning of the different tasks, inter-task contrastive learning (ICL) is presented to guide the model to learn task-wise features via absorbing complementary information from other tasks. Furthermore, considering the computational costs of the transformer, we propose HCT+ to integrate the spatial and temporal adapter to access competitive performance on substantially fewer tunable parameters. Extensive experiments on our cataract dataset and a publicly available endoscopic PSI-AVA dataset demonstrate the outstanding performance of our method, consistently exceeding the state-of-the-art methods by a large margin. The code is available athttps://github.com/Aurora-hao/HCT.

Anterior segment optical coherence tomography (AS-OCT) is a popular imaging technique that can directly visualize the anterior segment structures while inherent speckle noise severely impairs visual readability and subsequent clinical analysis. Though unpaired OCT image denoising algorithms have been developed to improve visual quality considering the limited supervised clinical data, preserving the edge structures while denoising remains challenging, especially in AS-OCT images with little hierarchy and low contrast. This work proposes an edge enhancement generative adversarial network ($E^{2}GAN$) based contrast-aware, particularly for unpaired AS-OCT image denoising. Specifically, to improve edge-structure consistency, we design a contrast attention mechanism for exploiting diverse hierarchical knowledge from multiple contrast images and adopt particular gradient-guided speckle filtering modules with an edge preservation loss for stabilizing the network. Additionally, considering that bi-directional GANs often focus on global appearance rather than essential features,$E^{2}GAN$adds a perceptual quality constraint into the cycle consistency. Extensive experiments validate the superiority of$E^{2}GAN$for AS-OCT image denoising and the benefits for downstream clinical analysis. Further experiments on the synthetic retinal OCT images prove the generalization of$E^{2}GAN$.

The morphologies of vessel-like structures, such as blood vessels and nerve fibres, play significant roles in disease diagnosis, e.g., Parkinson's disease. Although deep network-based refinement segmentation and topology-preserving segmentation methods recently have achieved promising results in segmenting vessel-like structures, they still face two challenges: 1) existing methods often have limitations in rehabilitating subsection ruptures in segmented vessel-like structures; 2) they are typically overconfident in predicted segmentation results. To tackle these two challenges, this paper attempts to leverage the potential of spatial interconnection relationships among subsection ruptures from the structure rehabilitation perspective. Based on this perspective, we propose a novel Vessel-like Structure Rehabilitation Network (VSR-Net) to both rehabilitate subsection ruptures and improve the model calibration based on coarse vessel-like structure segmentation results. VSR-Net first constructs subsection rupture clusters via a Curvilinear Clustering Module (CCM). Then, the well-designed Curvilinear Merging Module (CMM) is applied to rehabilitate the subsection ruptures to obtain the refined vessel-like structures. Extensive experiments on six 2D/3D medical image datasets show that VSR-Net significantly outperforms state-of-the-art (SOTA) refinement segmentation methods with lower calibration errors. Additionally, we provide quantitative analysis to explain the morphological difference between the VSR-Net's rehabilitation results and ground truth (GT), which are smaller compared to those between SOTA methods and GT, demonstrating that our method more effectively rehabilitates vessel-like structures.

Salient and small lesions (e.g., microaneurysms on fundus) both play significant roles in real-world disease diagnosis under medical image examinations. Although deep neural networks (DNNs) have achieved promising medical image classification performance, they often have limitations in capturing both salient and small lesion information, restricting performance improvement in imbalanced medical image classification. Recently, with the advent of DNN-based style transfer in medical image generation, the roles of clinical styles have attracted great interest, as they are crucial indicators of lesions. Motivated by this observation, we propose a novel Adaptive Dual-Axis Style-based Recalibration (ADSR) module, leveraging the potential of clinical styles to guide DNNs in effectively learning salient and small lesion information from a dual-axis perspective. ADSR first emphasizes salient lesion information via global style-based adaptation, then captures small lesion information with pixel-wise style-based fusion. We construct an ADSR-Net for imbalanced medical image classification by stacking multiple ADSR modules. Additionally, DNNs typically adopt cross-entropy loss for parameter optimization, which ignores the impacts of class-wise predicted probability distributions. To address this, we introduce a new Class-wise Statistics Loss (CWS) combined with CE to further boost imbalanced medical image classification results. Extensive experiments on five imbalanced medical image datasets demonstrate not only the superiority of ADSR-Net and CWS over state-of-the-art (SOTA) methods but also their improved confidence calibration results. For example, ADSR-Net with the proposed loss significantly outperforms CABNet50 by 21.39% and 27.82% in F1 and B-ACC while reducing 3.31% and 4.57% in ECE and BS on ISIC2018.

Early-stage diabetic retinopathy (DR) presents challenges in clinical diagnosis due to inconspicuous and minute microaneurysms (MAs), resulting in limited research in this area. Additionally, the potential of emerging foundation models, such as the segment anything model (SAM), in medical scenarios remains rarely explored. In this work, we propose a human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg, based on SAM. GlanceSeg enables real-time segmentation of MA lesions as ophthalmologists review fundus images. Our human-in-the-loop framework integrates the ophthalmologist's gaze maps, allowing for rough localization of minute lesions in fundus images. Subsequently, a saliency map is generated based on the located region of interest, which provides prompt points to assist the foundation model in efficiently segmenting MAs. Finally, a domain knowledge filtering (DKF) module refines the segmentation of minute lesions. We conducted experiments on two newly-built public datasets, i.e., IDRiD and Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg through visualized illustrations and quantitative measures. Additionally, we demonstrated that GlanceSeg improves annotation efficiency for clinicians and further enhances segmentation performance through fine-tuning using annotations. The clinician-friendly GlanceSeg is able to segment small lesions in real-time, showing potential for clinical applications.

Optical coherence tomography (OCT) is a widely used non-invasive imaging modality for ophthalmic diagnosis. However, the inherent speckle noise becomes the leading cause of OCT image quality, and efficient speckle removal algorithms can improve image readability and benefit automated clinical analysis. As an ill-posed inverse problem, it is of utmost importance for speckle removal to learn suitable priors. In this work, we develop a score prior guided iterative solver (SPIS) with logarithmic space to remove speckles in OCT images. Specifically, we model the posterior distribution of raw OCT images as a data consistency term and transform the speckle removal from a nonlinear into a linear inverse problem in the logarithmic domain. Subsequently, the learned prior distribution through the score function from the diffusion model is utilized as a constraint for the data consistency term into the linear inverse optimization, resulting in an iterative speckle removal procedure that alternates between the score prior predictor and the subsequent non-expansive data consistency corrector. Experimental results on the private and public OCT datasets demonstrate that the proposed SPIS has an excellent performance in speckle removal and out-of-distribution (OOD) generalization. Further downstream automatic analysis on the OCT images verifies that the proposed SPIS can benefit clinical applications.

Instrument-tissue interaction detection, a task aimed at understanding surgical scenes from videos, holds immense importance in constructing computer-assisted surgery systems. Existing methods for this task consist of two stages: instance detection and interaction prediction. This sequential and separate model structure limits both effectiveness and efficiency, making it difficult to deploy on surgical robotic platforms. In this paper, we propose an end-to-end Action-Instance Progressive Learning Network (AIPNet) for the task. The model operates in three steps: action detection, instance detection, and action class refinement. Starting with coarse-scale proposals, the model progressively refines them into coarse-grained actions, which then serve as proposals for instance detection. The action prediction results are further refined using instance features through late fusion. These progressive learning processes improve the performance of the end-to-end model. Additionally, we introduce Dynamic Proposal Generators (DPG) to create dynamic adaptive learnable proposals for each video frame. To address the training challenges of this multi-task model, semantic supervised training is introduced to transfer prior language knowledge, and a training label strategy is proposed to generate unrelated instrument-tissue pair labels for enhanced supervision. Experimental results on PhacoQ and CholecQ datasets show that the proposed method achieves superior accuracy and faster processing speed than state-of-the-art models.

Ocular surface diseases affect a significant portion of the population worldwide. Accurate segmentation and quantification of different ocular surface structures are crucial for the understanding of these diseases and clinical decision-making. However, the automated segmentation of the ocular surface structure is relatively unexplored and faces several challenges. Ocular surface structure boundaries are often inconspicuous and obscured by glare from reflections. In addition, the segmentation of different ocular structures always requires training of multiple individual models. Thus, developing a one-model-fits-all segmentation approach is desirable. In this paper, we introduce a randomness-restricted diffusion model for multiple ocular surface structure segmentation. First, a time-controlled fusion-attention module (TFM) is proposed to dynamically adjust the information flow within the diffusion model, based on the temporal relationships between the network's input and time. TFM enables the network to effectively utilize image features to constrain the randomness of the generation process. We further propose a low-frequency consistency filter and a new loss to alleviate model uncertainty and error accumulation caused by the multi-step denoising process. Extensive experiments have shown that our approach can segment seven different ocular surface structures. Our method performs better than both dedicated ocular surface segmentation methods and general medical image segmentation methods. We further validated the proposed method over two clinical datasets, and the results demonstrated that it is beneficial to clinical applications, such as the meibomian gland dysfunction grading and aqueous deficient dry eye diagnosis.

Cataract surgery, a widely performed operation worldwide, is incorporating semantic segmentation to advance computer-assisted intervention. However, the tissue appearance and illumination in cataract surgery often differ among clinical centers, intensifying the issue of domain shifts. While domain adaptation offers remedies to the shifts, the necessity for data centralization raises additional privacy concerns. To overcome these challenges, we propose a Multi-view Test-time Adaptation algorithm (MUTA) to segment cataract surgical scenes, which leverages multi-view learning to enhance model training within the source domain and model adaptation within the target domain. In the training phase, the segmentation model is equipped with multi-view decoders to boost its robustness against variations in cataract surgery. During the inference phase, test-time adaptation is implemented using multi-view knowledge distillation, enabling model updates in clinics without data centralization or privacy concerns. We conducted experiments in a simulated cross-center scenario using several cataract surgery datasets to evaluate the effectiveness of MUTA. Through comparisons and investigations, we have validated that MUTA effectively learns a robust source model and adapts the model to target data during the practical inference phase. Code and datasets are available at https://github.com/liamheng/CAI-algorithms.

Spatial attention (SA) mechanism has been widely incorporated into deep neural networks (DNNs), significantly lifting the performance in computer vision tasks via long-range dependency modeling. However, it may perform poorly in medical image analysis. Unfortunately, the existing efforts are often unaware that long-range dependency modeling has limitations in highlighting subtle lesion regions. To overcome this limitation, we propose a practical yet lightweight architectural unit, pyramid pixel context adaption (PPCA) module, which exploits multiscale pixel context information to recalibrate pixel position in a pixel-independent manner dynamically. PPCA first applies a well-designed cross-channel pyramid pooling (CCPP) to aggregate multiscale pixel context information, then eliminates the inconsistency among them by the well-designed pixel normalization (PN), and finally estimates per pixel attention weight via a pixel context integration. By embedding PPCA into a DNN with negligible overhead, the PPCA network (PPCANet) is developed for medical image classification. In addition, we introduce supervised contrastive learning to enhance feature representation by exploiting the potential of label information via supervised contrastive loss (CL). The extensive experiments on six medical image datasets show that the PPCANet outperforms state-of-the-art (SOTA) attention-based networks and recent DNNs. We also provide visual analysis and ablation study to explain the behavior of PPCANet in the decision-making process.

Object detection in aerial imagery presents a significant challenge due to large scale variations among objects. This paper proposes an evolutionary reinforcement learning agent, integrated within a coarse-to-fine object detection framework, to optimize the scale for more effective detection of objects in such images. Specifically, a set of patches potentially containing objects are first generated. A set of rewards measuring the localization accuracy, the accuracy of predicted labels, and the scale consistency among nearby patches are designed in the agent to guide the scale optimization. The proposed scale-consistency reward ensures similar scales for neighboring objects of the same category. Furthermore, a spatial-semantic attention mechanism is designed to exploit the spatial semantic relations between patches. The agent employs the proximal policy optimization strategy in conjunction with the evolutionary strategy, effectively utilizing both the current patch status and historical experience embedded in the agent. The proposed model is compared with state-of-the-art methods on two benchmark datasets for object detection on drone imagery. It significantly outperforms all the compared methods. Code is available at https://github.com/UNNC-CV/EvOD/.

Convolutional neural networks (CNNs) have long been the paradigm of choice for robust medical image processing (MIP). Therefore, it is crucial to effectively and efficiently deploy CNNs on devices with different computing capabilities to support computer-aided diagnosis. Many methods employ factorized convolutional layers to alleviate the burden of limited computational resources at the expense of expressiveness. To this end, given weak medical image-driven CNN model optimization, a Singular value equalization generalizer-induced Factorized Convolution (SFConv) is proposed to improve the expressive power of factorized convolutions in MIP models. We first decompose the weight matrix of convolutional filters into two low-rank matrices to achieve model reduction. Then minimize the KL divergence between the two low-rank weight matrices and the uniform distribution, thereby reducing the number of singular value directions with significant variance. Extensive experiments on fundus and OCTA datasets demonstrate that our SFConv yields competitive expressiveness over vanilla convolutions while reducing complexity.

The characteristics of 3D nodules on Computed Tomography (CT), including size, location, shape, and attenuation, are primary medical clues for distinguishing between benign and malignant nodules. To support evidence-based decision-making for lung cancer screening in clinical practice, we present a 3D Nodule Content-based Metric Learning (3D-NCML) network to retrieve subsolid-benign, subsolid-malignant, solid-benign, and solid-malignant nodules similar to the indeterminate ones. The inputs of 3D-NCML are 3D patches that exactly contain the whole nodule to ensure all visual information is included. A spatial position and size coding module, a shape encoder module, and an attenuation extraction module are designed based on medical clues for guiding the network to learn important characteristics of nodules. Experiments on the LIDC-IDRI dataset and a private dataset demonstrate that 3D-NCML outperforms other methods by quantitative and qualitative analysis, with more similar nodules retrieved and ranked ahead.

Pathology symptoms of Parkinson disease (PD) are different from those of retinal diseases in the retinal layers, which are subtle. However, segmenting pathology information of PD from retinal layers automatically based on optical coherence tomography (OCT) images has not been studied before. Although existing Transformer-based segmentation methods have achieved good segmentation results, they have limitations in capturing local context information. Convolutional neural networks (CNNs) can construct local context dependencies among pixels, which is complementary to Transformers. Particularly, edge information extraction is significant for accurate retinal layer segmentation, which is ignored by both Transformers and CNNs but can be captured by frequency domain learning methods. To fully leverage the advantages of Transformers, CNNs, and frequency domain learning methods, we propose a Wavelet Transformer (WaveFormer) for retinal layer segmentation based on OCT images. In the WaveFormer, we design a Wavelet Spatial Attention block to exploit the potential of frequency information. Based on these advantages, WaveFormer can be data-efficient in limited OCT images of PD. The experimental results on the OCT-PD segmentation dataset show that our WaveFormer outperforms existing Transformers and CNNs. For example, WaveFormer outperforms Swin-UNet by 3.41% of IoU.

Segment Routing (SR) is of great significance in the evolving landscape of Space-Air-Ground Integrated Networks (SAGIN). However, owing to hardware limitations and bandwidth constraints, directly applying existing SR-related solutions of terrestrial networks to satellite networks faces resource and performance challenges. Thus, in this paper, we introduce a novel framework called Satellite-Tailored Segment Routing (STSR) for satellite networks in SAGIN. We first propose the STSR data-plane protocol to provide lightweight source routing. Then, a routing policy integration scheme is proposed to support routing consistency in SAGIN. Simulation results reveal that compared to SRv6, in the case of 2000 satellites, STSR reduces routing path information in the header by 43.51 % and increases load efficiency by 43%-92%.

Although 2D radar signal representations, such as spectrograms and range-Doppler maps have been widely used for target recognition, 3D time-range-Doppler (TRD) has been less studied, partially because of the difficulties in extracting features from the TRD representation, i.e., shallow 3D neural networks have limited discriminant power, but repeatedly applying 3D convolutions will lead to an oversized 3D network. A hybrid 3D–2D network architecture, Progressively-Orthogonally-Mapped EfficientNet (POMEN), is proposed to address these challenges. More specifically, the proposed POMEN utilizes 3D convolutions in the earlier stages to capture the information embedded in the sparse 3D TRD representation, and to avoid the oversized feature map caused by excessively applying 3D convolutions, we propose to progressively map the 3D features into three sets of 2D features corresponding to the range-time signature, range-Doppler map and time-Doppler signature (spectrogram), respectively. Subsequently, 2D EfficientNet blocks were designed to extract discriminant information from the three sets of 2D feature maps. This hybrid 3D–2D network design effectively extracts features from the 3D TRD representation, thereby avoiding oversized features from full-sized 3D networks and the information loss of 2D networks on 2D representations. Finally, a homogeneous gated fusion network was designed to fuse the three sets of 2D features. The proposed method was evaluated on the UGRS, MIMOGR, and mmWRWD datasets. The experimental results for all datasets demonstrate that the proposed POMEN significantly and consistently outperforms the state-of-the-art models in both 2D and 3D representations.

Image segmentation is fundamental task for medical image analysis, whose accuracy is improved by the development of neural networks. However, the existing algorithms that achieve high-resolution performance require high-resolution input, resulting in substantial computational expenses and limiting their applicability in the medical field. Several studies have proposed dual-stream learning frameworks incorporating a super-resolution task as auxiliary. In this paper, we rethink these frameworks and reveal that the feature similarity between tasks is insufficient to constrain vessels or lesion segmentation in the medical field, due to their small proportion in the image. To address this issue, we propose a DS2F (Dual-Stream Shared Feature) framework, including a Shared Feature Extraction Module (SFEM). Specifically, we present Multi-Scale Cross Gate (MSCG) utilizing multi-scale features as a novel example of SFEM. Then we define a proxy task and proxy loss to enable the features focus on the targets based on the assumption that a limited set of shared features between tasks is helpful for their performance. Extensive experiments on six publicly available datasets across three different scenarios are conducted to verify the effectiveness of our framework. Furthermore, various ablation studies are conducted to demonstrate the significance of our DS2F.

Data distribution gaps often pose significant challenges to the use of deep segmentation models. However, retraining models for each distribution is expensive and time-consuming. In clinical contexts, device-embedded algorithms and networks, typically unretrainable and unaccessable post-manufacture, exacerbate this issue. Generative translation methods offer a solution to mitigate the gap by transferring data across domains. However, existing methods mainly focus on intensity distributions while ignoring the gaps due to structure disparities. In this paper, we formulate a new image-to-image translation task to reduce structural gaps. We propose a simple, yet powerful Structure-Unbiased Adversarial (SUA) network which accounts for both intensity and structural differences between the training and test sets for segmentation. It consists of a spatial transformation block followed by an intensity distribution rendering module. The spatial transformation block is proposed to reduce the structural gaps between the two images. The intensity distribution rendering module then renders the deformed structure to an image with the target intensity distribution. Experimental results show that the proposed SUA method has the capability to transfer both intensity distribution and structural content between multiple pairs of datasets and is superior to prior arts in closing the gaps for improving segmentation.

Robot-assisted catheterization offers a promising technique for cardiovascular interventions, addressing the limitations of manual interventional surgery, where precise tool manipulation is critical. In remote-control robotic systems, the lack of force feedback and imprecise navigation challenge cooperation between the surgeon and robot. This study proposes a manipulation-based evaluation framework to assess the cooperative performance between different operators and robot using kinesthetic, kinematic, and haptic data from multi-sensor technologies. The proposed evaluation framework achieves a recognition accuracy of 99.99% in assessing the cooperation between operator and robot. Additionally, the study investigates the impact of delay factors, considering no delay, constant delay, and variable delay, on cooperation characteristics. The findings suggest that variable delay contributes to improved cooperation performance between operator and robot in a primary-secondary isomorphic robotic system, compared to a constant delay factor. Furthermore, operators with experience in manual percutaneous coronary interventions exhibit significantly better cooperative manipulate on with the robot system than those without such experience, with respective synergy ratios of 89.66%, 90.28%, and 91.12% based on the three aspects of delay consideration. Moreover, the study explores interaction information, including distal force of tools-tissue and contact force of hand-control-ring, to understand how operators with different technical skills adjust their control strategy to prevent damage to the vascular vessel caused by excessive force while ensuring enough tension to navigate complex paths. The findings highlight the potential of variable delay to enhance cooperative control strategies in robotic catheterization systems, providing a basis for optimizing surgeon-robot collaboration in cardiovascular interventions.

Medical imaging provides many valuable clues involving anatomical structure and pathological characteristics. However, image degradation is a common issue in clinical practice, which can adversely impact the observation and diagnosis by physicians and algorithms. Although extensive enhancement models have been developed, these models require a well pre-training before deployment, while failing to take advantage of the potential value of inference data after deployment. In this paper, we raise an algorithm for source-free unsupervised domain adaptive medical image enhancement (SAME), which adapts and optimizes enhancement models using test data in the inference phase. A structure-preserving enhancement network is first constructed to learn a robust source model from synthesized training data. Then a teacher-student model is initialized with the source model and conducts source-free unsupervised domain adaptation (SFUDA) by knowledge distillation with the test data. Additionally, a pseudo-label picker is developed to boost the knowledge distillation of enhancement tasks. Experiments were implemented on ten datasets from three medical image modalities to validate the advantage of the proposed algorithm, and setting analysis and ablation studies were also carried out to interpret the effectiveness of SAME. The remarkable enhancement performance and benefits for downstream tasks demonstrate the potential and generalizability of SAME. The code is available at https://github.com/liamheng/Annotation-free-Medical-Image-Enhancement.

Instrument-tissue interaction detection task, which helps understand surgical activities, is vital for constructing computer-assisted surgery systems but with many challenges. Firstly, most models represent instrument-tissue interaction in a coarse-grained way which only focuses on classification and lacks the ability to automatically detect instruments and tissues. Secondly, existing works do not fully consider relations between intra- and inter-frame of instruments and tissues. In the paper, we propose to represent instrument-tissue interaction as 〈 instrument class, instrument bounding box, tissue class, tissue bounding box, action class 〉 quintuple and present an Instrument-Tissue Interaction Detection Network (ITIDNet) to detect the quintuple for surgery videos understanding. Specifically, we propose a Snippet Consecutive Feature (SCF) Layer to enhance features by modeling relationships of proposals in the current frame using global context information in the video snippet. We also propose a Spatial Corresponding Attention (SCA) Layer to incorporate features of proposals between adjacent frames through spatial encoding. To reason relationships between instruments and tissues, a Temporal Graph (TG) Layer is proposed with intra-frame connections to exploit relationships between instruments and tissues in the same frame and inter-frame connections to model the temporal information for the same instance. For evaluation, we build a cataract surgery video (PhacoQ) dataset and a cholecystectomy surgery video (CholecQ) dataset. Experimental results demonstrate the promising performance of our model, which outperforms other state-of-the-art models on both datasets.

Time-of-flight magnetic resonance angiography (TOF-MRA) is the least invasive and ionizing radiation-free approach for cerebrovascular imaging, but variations in imaging artifacts across different clinical centers and imaging vendors result in inter-site and inter-vendor heterogeneity, making its accurate and robust cerebrovascular segmentation challenging. Moreover, the limited availability and quality of annotated data pose further challenges for segmentation methods to generalize well to unseen datasets. In this paper, we construct the largest and most diverse TOF-MRA dataset (COSTA) from 8 individual imaging centers, with all the volumes manually annotated. Then we propose a novel network for cerebrovascular segmentation, namely CESAR, with the ability to tackle feature granularity and image style heterogeneity issues. Specifically, a coarse-to-fine architecture is implemented to refine cerebrovascular segmentation in an iterative manner. An automatic feature selection module is proposed to selectively fuse global long-range dependencies and local contextual information of cerebrovascular structures. A style self-consistency loss is then introduced to explicitly align diverse styles of TOF-MRA images to a standardized one. Extensive experimental results on the COSTA dataset demonstrate the effectiveness of our CESAR network against state-of-the-art methods. We have made 6 subsets of COSTA with the source code online available, in order to promote relevant research in the community.

The segmentation of blurred cell boundaries in cornea endothelium microscope images is challenging, which affects the clinical parameter estimation accuracy. Existing deep learning methods only consider pixel-wise classification accuracy and lack of utilization of cell structure knowledge. Therefore, the segmentation of the blurred cell boundary is discontinuous. This paper proposes a structural prior guided network (SPG-Net) for corneal endothelium cell segmentation. We first employ a hybrid transformer convolution backbone to capture more global context. Then, we use Feature Enhancement (FE) module to improve the representation ability of features and Local Affinity-based Feature Fusion (LAFF) module to propagate structural information among hierarchical features. Finally, we introduce the joint loss based on cross entropy and structure similarity index measure (SSIM) to supervise the training process under pixel and structure levels. We compare the SPG-Net with various state-of-the-art methods on four corneal endothelial datasets. The experiment results suggest that the SPG-Net can alleviate the problem of discontinuous cell boundary segmentation and balance the pixel-wise accuracy and structure preservation. We also evaluate the agreement of parameter estimation between ground truth and the prediction of SPG-Net. The statistical analysis results show a good agreement and correlation.

Intracranial Aneurysm (IA) lesion segmentation is significant for IA treatment, which is one of the high death rate and deformity cerebrovascular diseases. Segmenting the IA lesions accurately is still challenging in digital subtraction angiography (DSA) images due to blurred boundaries, imaging noise, and intracranial vascular morphologies. In this paper, we are the first time to propose a novel instance segmentation network architecture, Graph Mask2Former, to segment IA lesions automatically based on DSA images. Specifically, we apply a graph convolution module to reassign label information, aiming to adjust the confidence weight of error instances adaptively. Furthermore, we design a local refinement module to refine the coarse mask output. The extensive experiments on the clinical IA- DSA and LiTS datasets show that our method outperforms recent state-of-the-art methods. This paper also provides the visual analysis to explain the inherent behavior of our method.

In computer-assisted surgeries, monocular depth estimation plays an important role, which provides navigation for surgeons by computing precise depth information. In recent years, depth estimation has achieved significant breakthroughs with the application of deep learning. However, the lack of depth ground truth in the ophthalmology surgery scene has become an obstacle to the development of depth estimation in this scene. To resolve this problem, we built one synthetic dataset for cataract surgeries. The dataset contains information on RGB images, depth maps, and segmentation masks. We also adopt the state-of-the-art methods of depth estimation on this dataset as the baseline model to build the benchmark. We also analyze the generalization of the baseline models trained on the synthetic dataset to the real surgical scene.

Optical coherence tomography (OCT) imaging technique has been widely used for ocular disease diagnosis. However, speckles occur in OCT images due to the property of coherent imaging, inevitably affecting the visual quality and clinical analysis. To alleviate this problem, we propose a novel gradient-guided speckle image filtering method (GGSF) with structure enhancement for directly removing speckles in OCT images. Specifically, the multiplicative characteristic of speckle noise is incorporated into the guided filtering processing for modeling raw OCT images. To avoid getting trapped in image distortions, we further employ gradient regularization to integrate the structure prior information into the guided speckle image filtering procedure. Additionally, we introduce the statistical property of speckle noise obeying a gamma distribution into the least square method solver for the resulting non-convex GGSF model. Experimental results on the AS-OCT dataset demonstrate the effectiveness of GGSF for OCT image despeckling compared with competitive methods. Furthermore, we validate the benefits of GGSF for subsequent clinical analysis with the CM-OCT dataset.

Cataracts are the leading cause of visual impairment and blindness globally attracting abroad attention from society. Over the years researchers have developed many state-of-the-art convolutional neural networks (CNNs) to recognize cataract severity levels based on different ophthalmic images. However most current works focus on improving cataract recognition performance by designing complex CNNs often ignoring resource-constrained medical device limitations. To this problem this paper proposes a novel dual-mixed channel-independent convolution (DMIConv) method which takes advantage of the multiscale convolution kernels by combining a depthwise convolution with a depthwise dilated convolution sequentially. Moreover we build a lightweight dual-mixed channel-independent network (DMINet) to recognize cataracts. To verify the effectiveness and efficiency of DMINet we conduct extensive experiments on a clinical anterior segment optical coherence tomography (AS-OCT) dataset of nuclear cataract (NC) and a publicly available OCT dataset. The results show that our proposed DMINet keeps a better tradeoff between the model complexity and the classification performance than efficient CNNs e.g DMINet outperforms MixNet by 3.34% of accuracy by using 4.58 % fewer parameters

Corneal endothelial cell segmentation plays an important role in quantifying clinical indicators for the cornea health state evaluation. Although Convolution Neural Networks (CNNs) are widely used for medical image segmentation, their receptive fields are limited. Recently, Transformer outperforms convolution in modeling long-range dependencies but lacks local inductive bias so the pure transformer network is difficult to train on small medical image datasets. Moreover, Transformer networks cannot be effectively adopted for secular microscopes as they are parameter-heavy and computationally complex. To this end, we find that appropriately limiting attention spans and modeling information at different granularity can introduce local constraints and enhance attention representations. This paper explores a hierarchy full self-attention lightweight network for medical image segmentation, using Local and Global (LoGo) transformers to separately model attention representation at low-level and high-level layers. Specifically, the local efficient transformer (LoTr) layer is employed to decompose features into finer-grained elements to model local attention representation, while the global axial transformer (GoTr) is utilized to build long-range dependencies across the entire feature space. With this hierarchy structure, we gradually aggregate the semantic features from different levels efficiently. Experiment results on segmentation tasks of the corneal endothelial cell, the ciliary body, and the liver prove the accuracy, effectiveness, and robustness of our method. Compared with the convolution neural networks (CNNs) and the hybrid CNN-Transformer state-of-the-art (SOTA) methods, the LoGo transformer obtains the best result.

Abstract Despite impressive progress in face super‐resolution (SR), it is an open challenge to reconstruct a reliable SR face that preserves authentic facial characteristics. Here, the problem of super‐resolving low‐resolution (LR) faces to high‐resolution (HR) ones is addressed. To tackle the ill‐posed nature of face SR, the cascaded super‐resolution network (CSRNet) is proposed to utilize shape and identity priors jointly and progressively, the first to explore multiple priors. Specifically, CSRNet adopts a cascaded structure to transform an LR face to HR face progressively via multiple stages. At each stage, CSRNet forces its output face image to match both the shape priors and identity priors extracted from the ground‐truth HR face. The shape priors estimated in one stage are merged into the inputs of its subsequent stage to provide rich information for the face SR. To generate realistic yet discriminative faces, the cascaded super‐resolution generative adversarial network (CSRGAN) is also proposed to incorporate the adversarial loss and identification loss into CSRNet. Extensive experiments on popular benchmarks show that the CSRNet and CSRGAN outperform existing face SR state‐of‐the‐art methods, both quantitatively and qualitatively, and detailed ablation studies show the advantage of this method.

Ciliary muscle segmentation in Anterior Segment Optical Coherence Tomography (AS-OCT) images is critical significance, yet challenging due to ambiguous boundaries. In this paper, we propose a hierarchical attention multi-task network, HA-Net, based on U-Net for ciliary muscle segmentation using AS-OCT images. The network comprises a primary task for ciliary muscle segmentation and two auxiliary tasks for signed distance map regression and key point localization. The signed distance map is employed to incorporate shape priors into the model and delineate the ciliary muscle boundary, while key point localization guides the model to focus on ambiguous regions. Notably, in contrast to the widely-used multi-task model that generates results in parallel, we introduce a hierarchical attention module to exploit the affiliation prior of three tasks for generating outputs serially. Experimental results on CM544 dataset demonstrate that HA-Net outperforms state-of-the-art methods in ciliary muscle segmentation, with 0.9178 Dice score and 7.11 pixels HD95. Additionally, as a by-product of the multi-task model, key point localization facilitates the measurement of ciliary muscle thickness in clinical analysis.

Detecting 3D mask attacks to a face recognition system is challenging. Although genuine faces and 3D face masks show significantly different remote photoplethysmography (rPPG) signals, rPPG-based face anti-spoofing methods often suffer from performance degradation due to unstable face alignment in the video sequence and weak rPPG signals. To enhance the rPPG signal in a motion-robust way, a landmark-anchored face stitching method is proposed to align the faces robustly and precisely at the pixel-wise level by using both SIFT keypoints and facial landmarks. To better encode the rPPG signal, a weighted spatial-temporal representation is proposed, which emphasizes the face regions with rich blood vessels. In addition, characteristics of rPPG signals in different color spaces are jointly utilized. To improve the generalization capability, a lightweight EfficientNet with a Gated Recurrent Unit (GRU) is designed to extract both spatial and temporal features from the rPPG spatial-temporal representation for classification. The proposed method is compared with the state-of-the-art methods on five benchmark datasets under both intra-dataset and cross-dataset evaluations. The proposed method shows a significant and consistent improvement in performance over other state-of-the-art rPPG-based methods for face spoofing detection.

Multi-label image classification is a fundamental but challenging task in computer vision. To tackle the problem, the label-related semantic information is often exploited, but the background context and spatial semantic information of related objects are not fully utilized. To address these issues, a multi-branch deep neural network is proposed in this paper. The first branch is designed to extract the discriminant information from regions of interest to detect target objects. In the second branch, a spatial context-aware approach is proposed to better capture the contextual information of an object in its surroundings by using an adaptive patch expansion mechanism. It helps the detection of small objects that are easily lost without the support of context information. The third one, the object-attentional branch, exploits the spatial semantic relations between the target object and its related objects, to better detect partially occluded, small or dim objects with the support of those easily detectable objects. To better encode such relations, an attention mechanism jointly considering the spatial and semantic relations between objects is developed. Two widely used benchmark datasets for multi-labeling classification, MS COCO and PASCAL VOC, are used to evaluate the proposed framework. The experimental results demonstrate that the proposed method outperforms the state-of-the-art methods for multi-label image classification.

Few-shot learning (FSL) is promising in the field of medical image analysis due to high cost of establishing high-quality medical datasets. Many FSL approaches have been proposed in natural image scenes. However, present FSL methods are rarely evaluated on medical images and the FSL technology applicable to medical scenarios need to be further developed. Meta-learning has supplied an optional framework to address the challenging FSL setting. In this paper, we propose a novel multi-learner based FSL method for multiple medical image classification tasks, combining meta-learning with transfer-learning and metric-learning. Our designed model is composed of three learners, including auto-encoder, metric-learner and task-learner. In transfer-learning, all the learners are trained on the base classes. In the ensuing meta-learning, we leverage multiple novel tasks to fine-tune the metric-learner and task-learner in order to fast adapt to unseen tasks. Moreover, to further boost the learning efficiency of our model, we devised real-time data augmentation and dynamic Gaussian disturbance soft label (GDSL) scheme as effective generalization strategies of few-shot classification tasks. We have conducted experiments for three-class few-shot classification tasks on three newly-built challenging medical benchmarks, BLOOD, PATH and CHEST. Extensive comparisons to related works validated that our method achieved top performance both on homogeneous medical datasets and cross-domain datasets.

The growth rate of pulmonary nodules is a critical clue to the cancerous diagnosis. It is essential to monitor their dynamic progressions during pulmonary nodule management. To facilitate the prosperity of research on nodule growth prediction, we organized and published a temporal dataset called NLSTt with consecutive computed tomography (CT) scans. Based on the self-built dataset, we develop a visual learner to predict the growth for the following CT scan qualitatively and further propose a model to predict the growth rate of pulmonary nodules quantitatively, so that better diagnosis can be achieved with the help of our predicted results. To this end, in this work, we propose a parameterized Gempertz-guided morphological autoencoder (GM-AE) to generate any future-time-span high-quality visual appearances of pulmonary nodules from the baseline CT scan. Specifically, we parameterize a popular mathematical model for tumor growth kinetics, Gompertz, to predict future masses and volumes of pulmonary nodules. Then, we exploit the expected growth rate on the mass and volume to guide decoders generating future shape and texture of pulmonary nodules. We introduce two branches in an autoencoder to encourage shape-aware and textural-aware representation learning and integrate the generated shape into the textural-aware branch to simulate the future morphology of pulmonary nodules. We conduct extensive experiments on the self-built NLSTt dataset to demonstrate the superiority of our GM-AE to its competitive counterparts. Experiment results also reveal the learnable Gompertz function enjoys promising descriptive power in accounting for inter-subject variability of the growth rate for pulmonary nodules. Besides, we evaluate our GM-AE model on an in-house dataset to validate its generalizability and practicality. We make its code publicly available along with the published NLSTt dataset.

Despite the tremendous progress recently achieved by deep learning (DL) in medical image analysis, most DL models only concentrate on single data distribution, which follows the independent and identically distributed (i.i.d) assumption. However, in practice, image data distribution changes with clinical conditions, such as different scanner manufacturers, imaging settings, and statistics regions. Although one can further train the model on new data samples, updating a model with data from an unknown distribution will always result in the model’s performance degradation on the learned data, a notorious phenomenon called catastrophic forgetting. Therefore affects the applicability of DL algorithms in continuously changing clinical scenarios. In this study, we have proposed a new method to address the impact of changing distributions in continual learning scenarios and alleviate catastrophic forgetting. A gradient regularization approach is used to suppress forgetting, and a replay-oriented consistency calculation method combined with a subspace weighting strategy is proposed to improve the model plasticity further. The proposed replay-oriented gradient projection memory (RO-GPM) is evaluated on multiple fundus disease diagnosis datasets including a real-world application and a continual learning benchmark. The quantitative and visualization results demonstrate that the proposed RO-GPM achieves superior performance to state-of-the-art algorithms by a large margin.1

Existing multi-view learning methods based on the information bottleneck principle exhibit impressing generalization by capturing inter-view consistency and complementarity. They leverage cross-view joint information (consistency) and view-specific information (complementarity) while discarding redundant information. By fusing visual features, multi-view learning methods help medical image processing to produce more reliable predictions. However, multi-views of medical images often have low consistency and high complementarity due to modal differences in imaging or different projection depths, thus challenging existing methods to balance them to the maximal extent. To mitigate such an issue, we improve the information bottleneck (IB) loss function with a balanced regularization term, termed IBB loss, reassembling the constraints of multi-view consistency and complementarity. In particular, the balanced regularization term with a unique trade-off factor in IBB loss helps minimize the mutual information on consistency and complementarity to strike a balance. In addition, we devise a triplet multi-view network named TM net to learn the consistent and complementary features from multi-view medical images. By evaluating two datasets, we demonstrate the superiority of our method against several counterparts. The extensive experiments also confirm that our IBB loss significantly improves multi-view learning in medical images.

It turns out that convolutional neural networks (CNNs) have excellent medical image processing capabilities. Hence, effectively and efficiently deploying CNNs on devices with varying computing power to make computer-aided diagnosis puts on the agenda. However, it is a dilemma to balance the limited computing resources and model complexity. Previously, we proposed factorized convolution with spectral normalization (FConvSN) to mitigate the bottleneck of deploying CNNs for 2D medical images. But due to the cube structure of 3D convolutional kernels, it does not work well for 3D medical images. Directly flattening 3D kernels to 2D weights for matrix factorization may undermine the learning ability along depth-wise, resulting in the loss of depth information and the decline of model performance. To this end, we factorize a 3D convolutional kernel to 2D weight matrices with depth-wise dimensions, then assign an attentive score for each 2D weight matrix by a depth-wise dependencies-induced adaptive attention block (AA). AA with a temperature hyper-parameter helps convolution kernel to better capture depth-wise dependencies in 3D medical images, improving its learning ability along the depth direction. We term this novel factorized convolution as FConvAA used for compressing model complexity without impairing the depth-wise expressivity. We also impose spectral normalization (SN) for FConvAA to constrain spectral norm-wise weights. We conduct extensive experiments on the public lung CT dataset LUNA16 and the private retina OCT dataset to demonstrate the effectiveness and feasibility of our FConvAA.

Hard exudates (HE) is the most specific biomarker for retina edema. Precise HE segmentation is vital for disease diagnosis and treatment, but automatic segmentation is challenged by its large variation of characteristics including size, shape and position, which makes it difficult to detect tiny lesions and lesion boundaries. Considering the complementary features between segmentation and super-resolution tasks, this paper proposes a novel hard exudates segmentation method named SSMAF with an auxiliary super-resolution task, which brings in helpful detailed features for tiny lesion and boundaries detection. Specifically, we propose a fusion module named Multi-scale Attention Fusion (MAF) module for our dual-stream framework to effectively integrate features of the two tasks. MAF first adopts split spatial convolutional (SSC) layer for multi-scale features extraction and then utilize attention mechanism for features fusion of the two tasks. Considering pixel dependency, we introduce region mutual information (RMI) loss to optimize MAF module for tiny lesions and boundary detection. We evaluate our method on two public lesion datasets, IDRiD and E-Ophtha. Our method shows competitive performance with low-resolution inputs, both quantitatively and qualitatively. On E-Ophtha dataset, the method can achieve $\ge 3$% higher dice and recall compared with the state-of-the-art methods.

Multi-label image classification is a fundamental but challenging task in computer vision. Over the past few decades, solutions exploring relationships between semantic labels have made great progress. However, the underlying spatial-contextual information of labels is under-exploited. To tackle this problem, a spatial-context-aware deep neural network is proposed to predict labels taking into account both semantic and spatial information. This proposed framework is evaluated on Microsoft COCO and PASCAL VOC, two widely used benchmark datasets for image multi-labelling. The results show that the proposed approach is superior to the state-of-the-art solutions on dealing with the multi-label image classification problem.

Nuclear cataract (NC) is a prior age-related disease for blindness and vision impairment globally. Anterior segment optical coherence tomography (AS-OCT) image is a new ophthalmology image, which can capture the lens nucleus region clearly compared with other ophthalmic images, e.g., slit lamp images. Clinical research has suggested that features e.g., mean from AS-OCT images have varying correlations with NC severity levels. However, existing convolutional neural network (CNN) based NC classification works have not incorporated the clinical features into the network design to improve the performance. To this end, we propose a novel channel-wise and spatial feature recalibration network (CSFR-Net) to predict NC severity levels automatically, which is built on a stack of channel-wise and spatial feature recalibration (CSFR) modules. In each CSFR module, we construct a channel-wise feature recalibration block and a spatial feature recalibration block to recalibrate intermediate feature maps dynamically. This feature recalibration strategy enables CSFR-Net to highlight feature representations and suppress unnecessary ones in a global-and-local manner. We conduct extensive experiments on a clinical AS-OCT image dataset and CIFAR benchmarks. The results show that our CSFR-Net achieves better performance than state-of-the-art methods with less model complexity.

Nuclear cataract (NC) is a leading eye disease for blindness and vision impairment globally. Accurate and objective NC grading/classification is essential for clinically early intervention and cataract surgery planning. Anterior segment optical coherence tomography (AS-OCT) images are capable of capturing the nucleus region clearly and measuring the opacity of NC quantitatively. Recently, clinical research has suggested that the opacity correlation and repeatability between NC severity levels and the average nucleus density on AS-OCT images is high with the interclass and intraclass analysis. Moreover, clinical research has suggested that opacity distribution is uneven on the nucleus region, indicating that the opacities from different nucleus regions may play different roles in NC diagnosis. Motivated by the clinical priors, this paper proposes a simple yet effective region-based integration-and-recalibration attention (RIR), which integrates multiple feature map region representations and recalibrates the weights of each region via softmax attention adaptively. This region recalibration strategy enables the network to focus on high contribution region representations and suppress less useful ones. We combine the RIR block with the residual block to form a Residual-RIR module, and then a sequence of Residual-RIR modules are stacked to a deep network named region-based integration-and-recalibration network (RIR-Net), to predict NC severity levels automatically. The experiments on a clinical AS-OCT image dataset and two OCT datasets demonstrate that our method outperforms strong baselines and previous state-of-the-art methods. Furthermore, attention weight visualization analysis and ablation studies verify the capability of our RIR-Net for adjusting the relative importance of different regions in feature maps dynamically, agreeing with the clinical research.

Automatic angle-closure assessment in Anterior Segment OCT (AS-OCT) images is an important task for the screening and diagnosis of glaucoma, and the most recent computer-aided models focus on a binary classification of anterior chamber angles (ACA) in AS-OCT, i.e., open-angle and angle-closure. In order to assist clinicians who seek better to understand the development of the spectrum of glaucoma types, a more discriminating three-class classification scheme was suggested, i.e., the classification of ACA was expended to include open-, appositional- and synechial angles. However, appositional and synechial angles display similar appearances in an AS-OCT image, which makes classification models struggle to differentiate angle-closure subtypes based on static AS-OCT images. In order to tackle this issue, we propose a 2D-3D Hybrid Variation-aware Network (HV-Net) for open-appositional-synechial ACA classification from AS-OCT imagery. Specifically, taking into account clinical priors, we first reconstruct the 3D iris surface from an AS-OCT sequence, and obtain the geometrical characteristics necessary to provide global shape information. 2D AS-OCT slices and 3D iris representations are then fed into our HV-Net to extract cross-sectional appearance features and iris morphological features, respectively. To achieve similar results to those of dynamic gonioscopy examination, which is the current gold standard for diagnostic angle assessment, the paired AS-OCT images acquired in dark and light illumination conditions are used to obtain an accurate characterization of configurational changes in ACAs and iris shapes, using a Variation-aware Block. In addition, an annealing loss function was introduced to optimize our model, so as to encourage the sub-networks to map the inputs into the more conducive spaces to extract dark-to-light variation representations, while retaining the discriminative power of the learned features. The proposed model is evaluated across 1584 paired AS-OCT samples, and it has demonstrated its superiority in classifying open-, appositional- and synechial angles.

Automated detection of retinal structures, such as retinal vessels (RV), the foveal avascular zone (FAZ), and retinal vascular junctions (RVJ), are of great importance for understanding diseases of the eye and clinical decision-making. In this paper, we propose a novel Voting-based Adaptive Feature Fusion multi-task network (VAFF-Net) for joint segmentation, detection, and classification of RV, FAZ, and RVJ in optical coherence tomography angiography (OCTA). A task-specific voting gate module is proposed to adaptively extract and fuse different features for specific tasks at two levels: features at different spatial positions from a single encoder, and features from multiple encoders. In particular, since the complexity of the microvasculature in OCTA images makes simultaneous precise localization and classification of retinal vascular junctions into bifurcation/crossing a challenging task, we specifically design a task head by combining the heatmap regression and grid classification. We take advantage of three different en face angiograms from various retinal layers, rather than following existing methods that use only a single en face. We carry out extensive experiments on three OCTA datasets acquired using different imaging devices, and the results demonstrate that the proposed method performs on the whole better than either the state-of-the-art single-purpose methods or existing multi-task learning solutions. We also demonstrate that our multi-task learning method generalizes across other imaging modalities, such as color fundus photography, and may potentially be used as a general multi-task learning tool. We also construct three datasets for multiple structure detection, and part of these datasets with the source code and evaluation benchmark have been released for public access.

Cataracts are the leading cause of vision loss worldwide. Restoration algorithms are developed to improve the readability of cataract fundus images in order to increase the certainty in diagnosis and treatment for cataract patients. Unfortunately, the requirement of annotation limits the application of these algorithms in clinics. This paper proposes a network to annotation-freely restore cataractous fundus images (ArcNet) so as to boost the clinical practicability of restoration. Annotations are unnecessary in ArcNet, where the high-frequency component is extracted from fundus images to replace segmentation in the preservation of retinal structures. The restoration model is learned from the synthesized images and adapted to real cataract images. Extensive experiments are implemented to verify the performance and effectiveness of ArcNet. Favorable performance is achieved using ArcNet against state-of-the-art algorithms, and the diagnosis of ocular fundus diseases in cataract patients is promoted by ArcNet. The capability of properly restoring cataractous images in the absence of annotated data promises the proposed algorithm outstanding clinical practicability.

Accurate estimation and quantification of the corneal nerve fiber tortuosity in corneal confocal microscopy (CCM) is of great importance for disease understanding and clinical decision-making. However, the grading of corneal nerve tortuosity remains a great challenge due to the lack of agreements on the definition and quantification of tortuosity. In this paper, we propose a fully automated deep learning method that performs image-level tortuosity grading of corneal nerves, which is based on CCM images and segmented corneal nerves to further improve the grading accuracy with interpretability principles. The proposed method consists of two stages: 1) A pre-trained feature extraction backbone over ImageNet is fine-tuned with a proposed novel bilinear attention (BA) module for the prediction of the regions of interest (ROIs) and coarse grading of the image. The BA module enhances the ability of the network to model long-range dependencies and global contexts of nerve fibers by capturing second-order statistics of high-level features. 2) An auxiliary tortuosity grading network (AuxNet) is proposed to obtain an auxiliary grading over the identified ROIs, enabling the coarse and additional gradings to be finally fused together for more accurate final results. The experimental results show that our method surpasses existing methods in tortuosity grading, and achieves an overall accuracy of 85.64% in four-level classification. We also validate it over a clinical dataset, and the statistical analysis demonstrates a significant difference of tortuosity levels between healthy control and diabetes group. We have released a dataset with 1500 CCM images and their manual annotations of four tortuosity levels for public access. The code is available at: https://github.com/iMED-Lab/TortuosityGrading.

Anomaly detection in medical images refers to the identification of abnormal images with only normal images in the training set. Most existing methods solve this problem with a self-reconstruction framework, which tends to learn an identity mapping and reduces the sensitivity to anomalies. To mitigate this problem, in this paper, we propose a novel Proxy-bridged Image Reconstruction Network (ProxyAno) for anomaly detection in medical images. Specifically, we use an intermediate proxy to bridge the input image and the reconstructed image. We study different proxy types, and we find that the superpixel-image (SI) is the best one. We set all pixels' intensities within each superpixel as their average intensity, and denote this image as SI. The proposed ProxyAno consists of two modules, a Proxy Extraction Module and an Image Reconstruction Module. In the Proxy Extraction Module, a memory is introduced to memorize the feature correspondence for normal image to its corresponding SI, while the memorized correspondence does not apply to the abnormal images, which leads to the information loss for abnormal image and facilitates the anomaly detection. In the Image Reconstruction Module, we map an SI to its reconstructed image. Further, we crop a patch from the image and paste it on the normal SI to mimic the anomalies, and enforce the network to reconstruct the normal image even with the pseudo abnormal SI. In this way, our network enlarges the reconstruction error for anomalies. Extensive experiments on brain MR images, retinal OCT images and retinal fundus images verify the effectiveness of our method for both image-level and pixel-level anomaly detection.

This work focuses on image anomaly detection by leveraging only normal images in the training phase. Most previous methods tackle anomaly detection by reconstructing the input images with an autoencoder (AE)-based model, and an underlying assumption is that the reconstruction errors for the normal images are small, and those for the abnormal images are large. However, these AE-based methods, sometimes, even reconstruct the anomalies well; consequently, they are less sensitive to anomalies. To conquer this issue, we propose to reconstruct the image by leveraging the structure-texture correspondence. Specifically, we observe that, usually, for normal images, the texture can be inferred from its corresponding structure (e.g., the blood vessels in the fundus image and the structured anatomy in optical coherence tomography image), while it is hard to infer the texture from a destroyed structure for the abnormal images. Therefore, a structure-texture correspondence memory (STCM) module is proposed to reconstruct image texture from its structure, where a memory mechanism is used to characterize the mapping from the normal structure to its corresponding normal texture. As the correspondence between destroyed structure and texture cannot be characterized by the memory, the abnormal images would have a larger reconstruction error, facilitating anomaly detection. In this work, we utilize two kinds of complementary structures (i.e., the semantic structure with human-labeled category information and the low-level structure with abundant details), which are extracted by two structure extractors. The reconstructions from the two kinds of structures are fused together by a learned attention weight to get the final reconstructed image. We further feed the reconstructed image into the two aforementioned structure extractors to extract structures. On the one hand, constraining the consistency between the structures extracted from the original input and that from the reconstructed image would regularize the network training; on the other hand, the error between the structures extracted from the original input and that from the reconstructed image can also be used as a supplement measurement to identify the anomaly. Extensive experiments validate the effectiveness of our method for image anomaly detection on both industrial inspection images and medical images.

Multimedia information processing course includes image processing, text processing, video processing, audio processing, graphics, and animation. Classical multimedia information processing course is to lecture these contents as independent course units, making the course teaching inconsistently and student's learning interest and attention lost easily. Archaeology is a cross-disciplinary field, where an archaeological research project needs to use a variety of multimedia information processing technologies. This paper introduces a novel cross-disciplinary course teaching approach to combine traditional multimedia information processing techniques with archaeological research content in the new engineering era, which increases students' attention and interest in the multimedia information processing course. Teaming up with the archaeology professor, we present two multimedia-archaeology projects: intelligent pottery fragments splicing and intelligent Oracle inscription recognition. We have addressed a few challenges in our courses: 1) how to guide students efficiently to implement different project contents, e.g., using the scanner to acquire three-dimensional porcelain fragment data skillfully. 2) How to inspire students to learn and use various multimedia processing technologies and archaeology knowledge. 3) How the teacher adapts the teaching content according to the dynamic interests of the students. To address these challenges, students are grouped into two project teams based on their strengths and interests. We introduce collaborative learning and active learning strategies to help students learn and use different knowledge and address project problems and learning problems. We also invite the archaeological professor to teach basic archaeology knowledge in the class. Furthermore, to better understand the students' learning situation, we present a weekly project progress report approach, which can also help the teacher adjust the teaching content. This teaching approach can enhance the continuity of multimedia information processing teaching and stimulate students' enthusiasm and creativity in learning. Moreover, it can deepen the cultural atmosphere of the teaching in an engineering course.

Face spoofing detection against paper attack and video-replay attack has been well studied, whereas detecting 3D face mask attack remains challenging. Remote photoplethysmography (rPPG) signal is a recently developed liveness clue for face-spoofing detection. The main challenge of existing rPPG-based methods is that the signal can be easily distorted by background noise or object motion. To address this problem, in this work, we propose an rPPG-based face-spoofing detection method using multiple regions of interests (ROIs) covering entire face, and emphasize the regions containing richer rPPG signals using larger weights. The rPPG signals of these regions form a weighted spatial-temporal map. In view of the discriminant power of EfficientNet over other deep convolutional neural networks, we propose a domain-specific EfficientNet as the classification method. Extensive experiments on two databases namely 3DMAD and HKBU-Mars V2 demonstrate the superior performance of the proposed method over state-of-the-art rPPG-based face-spoofing-detection algorithms.

When encountering a dubious diagnostic case, medical instance retrieval can help radiologists make evidence-based diagnoses by finding images containing instances similar to a query case from a large image database. The similarity between the query case and retrieved similar cases is determined by visual features extracted from pathologically abnormal regions. However, the manifestation of these regions often lacks specificity, i.e., different diseases can have the same manifestation, and different manifestations may occur at different stages of the same disease. To combat the manifestation ambiguity in medical instance retrieval, we propose a novel deep framework called Y-Net, encoding images into compact hash-codes generated from convolutional features by feature aggregation. Y-Net can learn highly discriminative convolutional features by unifying the pixel-wise segmentation loss and classification loss. The segmentation loss allows exploring subtle spatial differences for good spatial-discriminability while the classification loss utilizes class-aware semantic information for good semantic-separability. As a result, Y-Net can enhance the visual features in pathologically abnormal regions and suppress the disturbing of the background during model training, which could effectively embed discriminative features into the hash-codes in the retrieval stage. Extensive experiments on two medical image datasets demonstrate that Y-Net can alleviate the ambiguity of pathologically abnormal regions and its retrieval performance outperforms the state-of-the-art method by an average of 9.27% on the returned list of 10.

Optical Coherence Tomography Angiography (OCTA) is a non-invasive imaging technique that has been increasingly used to image the retinal vasculature at capillary level resolution. However, automated segmentation of retinal vessels in OCTA has been under-studied due to various challenges such as low capillary visibility and high vessel complexity, despite its significance in understanding many vision-related diseases. In addition, there is no publicly available OCTA dataset with manually graded vessels for training and validation of segmentation algorithms. To address these issues, for the first time in the field of retinal image analysis we construct a dedicated Retinal OCTA SEgmentation dataset (ROSE), which consists of 229 OCTA images with vessel annotations at either centerline-level or pixel level. This dataset with the source code has been released for public access to assist researchers in the community in undertaking research in related topics. Secondly, we introduce a novel split-based coarse-to-fine vessel segmentation network for OCTA images (OCTA-Net), with the ability to detect thick and thin vessels separately. In the OCTA-Net, a split-based coarse segmentation module is first utilized to produce a preliminary confidence map of vessels, and a split-based refined segmentation module is then used to optimize the shape/contour of the retinal microvasculature. We perform a thorough evaluation of the state-of-the-art vessel segmentation models and our OCTA-Net on the constructed ROSE dataset. The experimental results demonstrate that our OCTA-Net yields better vessel segmentation performance in OCTA than both traditional and other deep learning methods. In addition, we provide a fractal dimension analysis on the segmented microvasculature, and the statistical analysis demonstrates significant differences between the healthy control and Alzheimer's Disease group. This consolidates that the analysis of retinal microvasculature may offer a new scheme to study various neurodegenerative diseases.

The development of medical imaging techniques has greatly supported clinical decision making. However, poor imaging quality, such as non-uniform illumination or imbalanced intensity, brings challenges for automated screening, analysis and diagnosis of diseases. Previously, bi-directional GANs (e.g., CycleGAN), have been proposed to improve the quality of input images without the requirement of paired images. However, these methods focus on global appearance, without imposing constraints on structure or illumination, which are essential features for medical image interpretation. In this paper, we propose a novel and versatile bi-directional GAN, named Structure and illumination constrained GAN (StillGAN), for medical image quality enhancement. Our StillGAN treats low- and high-quality images as two distinct domains, and introduces local structure and illumination constraints for learning both overall characteristics and local details. Extensive experiments on three medical image datasets (e.g., corneal confocal microscopy, retinal color fundus and endoscopy images) demonstrate that our method performs better than both conventional methods and other deep learning-based methods. In addition, we have investigated the impact of the proposed method on different medical image analysis and clinical tasks such as nerve segmentation, tortuosity grading, fovea localization and disease classification.

Deep hashing methods have been proved to be effective for the large-scale medical image search assisting reference-based diagnosis for clinicians. However, when the salient region plays a maximal discriminative role in ophthalmic image, existing deep hashing methods do not fully exploit the learning ability of the deep network to capture the features of salient regions pointedly. The different grades or classes of ophthalmic images may be share similar overall performance but have subtle differences that can be differentiated by mining salient regions. To address this issue, we propose a novel end-to-end network, named Attention-based Saliency Hashing (ASH), for learning compact hash-code to represent ophthalmic images. ASH embeds a spatial-attention module to focus more on the representation of salient regions and highlights their essential role in differentiating ophthalmic images. Benefiting from the spatial-attention module, the information of salient regions can be mapped into the hash-code for similarity calculation. Extensive experiments on two different modalities of ophthalmic image datasets demonstrate that the proposed ASH can further improve the retrieval performance compared to the state-of-the-art deep hashing methods due to the huge contributions of the spatial-attention module.

Latent Factor Model (LFM) is one of the most successful methods for Collaborative filtering (CF) in the recommendation system, in which both users and items are projected into a joint latent factor space. Base on matrix factorization applied usually in pattern recognition, LFM models user-item interactions as inner products of factor vectors of user and item in that space and can be efficiently solved by least square methods with optimal estimation. However, such optimal estimation methods are prone to overfitting due to the extreme sparsity of user-item interactions. In this paper, we propose a Bayesian treatment for LFM, named Bayesian Latent Factor Model (BLFM). Based on observed user-item interactions, we build a probabilistic factor model in which the regularization is introduced via placing prior constraint on latent factors, and the likelihood function is established over observations and parameters. Then we draw samples of latent factors from the posterior distribution with Variational Inference (VI) to predict expected value. We further make an extension to BLFM, called BLFMBias, incorporating user-dependent and item-dependent biases into the model for enhancing performance. Extensive experiments on the movie rating dataset show the effectiveness of our proposed models by compared with several strong baselines.

Nuclear cataract is one of the most common types of cataract. In the recent, ophthalmologists are increasingly using anterior segment optical coherence tomography (AS-OCT) images to diagnose many ocular diseases including cataract. The relationship between cataract and the lens opacity based on AS-OCT images has been being studied in clinical pioneer research. However, using AS-OCT images to classify cataract automatically based on computer-aided diagnosis (CAD) technique has not been seriously studied. This paper proposes a novel Convolutional Neural Network (CNN) model named GraNet for nuclear cataract classification based on AS-OCT images. In the GraNet, we introduce a grading block to learn high-level feature representations based on the pointwise convolution method. To further improve the classification performance, we propose a simple and efficient cross-training method is comprised of focal loss and cross-entropy loss. Extensive experiments are conducted on the AS-OCT image dataset, the results demonstrate that the proposed methods achieve better nuclear cataract classification results than baselines.

Irreversible visual impairment is often caused by primary angle-closure glaucoma, which could be detected via anterior segment optical coherence tomography (AS-OCT). In this paper, an automated system based on deep learning is presented for angle-closure detection in AS-OCT images. Our system learns a discriminative representation from training data that captures subtle visual cues not modeled by handcrafted features. A multilevel deep network is proposed to formulate this learning, which utilizes three particular AS-OCT regions based on clinical priors: 1) the global anterior segment structure; 2) local iris region; and 3) anterior chamber angle (ACA) patch. In our method, a sliding window-based detector is designed to localize the ACA region, which addresses ACA detection as a regression task. Then, three parallel subnetworks are applied to extract AS-OCT representations for the global image and at clinically relevant local regions. Finally, the extracted deep features of these subnetworks are concatenated into one fully connected layer to predict the angle-closure detection result. In the experiments, our system is shown to surpass previous detection methods and other deep learning systems on two clinical AS-OCT datasets.

The choroid provides oxygen and nourishment to the outer retina thus is related to the pathology of various ocular diseases. Optical coherence tomography (OCT) is advantageous in visualizing and quantifying the choroid in vivo. However, its application in the study of the choroid is still limited for two reasons. (1) The lower boundary of the choroid (choroid-sclera interface) in OCT is fuzzy, which makes the automatic segmentation difficult and inaccurate. (2) The visualization of the choroid is hindered by the vessel shadows from the superficial layers of the inner retina. In this paper, we propose to incorporate medical and imaging prior knowledge with deep learning to address these two problems. We propose a biomarker-infused global-to-local network (Bio-Net) for the choroid segmentation, which not only regularizes the segmentation via predicted choroid thickness, but also leverages a global-to-local segmentation strategy to provide global structure information and suppress overfitting. For eliminating the retinal vessel shadows, we propose a deep-learning pipeline, which firstly locate the shadows using their projection on the retinal pigment epithelium layer, then the contents of the choroidal vasculature at the shadow locations are predicted with an edge-to-texture generative adversarial inpainting network. The results show our method outperforms the existing methods on both tasks. We further apply the proposed method in a clinical prospective study for understanding the pathology of glaucoma, which demonstrates its capacity in detecting the structure and vascular changes of the choroid related to the elevation of intra-ocular pressure.

In the above article[1],Tables II,III, andVandFig. 6are incorrect. The correct images are provided below:

The detection of retinal vessel is of great importance in the diagnosis and treatment of many ocular diseases. Many methods have been proposed for vessel detection. However, most of the algorithms neglect the connectivity of the vessels, which plays an important role in the diagnosis. In this paper, we propose a novel method for retinal vessel detection. The proposed method includes a dense dilated network to get an initial detection of the vessels and a probability regularized walk algorithm to address the fracture issue in the initial detection. The dense dilated network integrates newly proposed dense dilated feature extraction blocks into an encoder-decoder structure to extract and accumulate features at different scales. A multi-scale Dice loss function is adopted to train the network. To improve the connectivity of the segmented vessels, we also introduce a probability regularized walk algorithm to connect the broken vessels. The proposed method has been applied on three public data sets: DRIVE, STARE and CHASE_DB1. The results show that the proposed method outperforms the state-of-the-art methods in accuracy, sensitivity, specificity and also area under receiver operating characteristic curve.

Machine learning has been widely used in medical image analysis under an assumption that the training and test data are under the same feature distributions. However, medical images from difference devices or the same device with different parameter settings are often contaminated with different amount and types of noises, which violate the above assumption. Therefore, the models trained using data from one device or setting often fail to work for that from another. Moreover, it is very expensive and tedious to label data and re-train models for all different devices or settings. To overcome this noise adaptation issue, it is necessary to leverage on the models trained with data from one device or setting for new data. In this paper, we reformulate this noise adaptation task as an image-to-image translation task such that the noise patterns from the test data are modified to be similar to those from the training data while the contents of the data are unchanged. In this paper, we propose a novel Noise Adaptation Generative Adversarial Network (NAGAN), which contains a generator and two discriminators. The generator aims to map the data from source domain to target domain. Among the two discriminators, one discriminator enforces the generated images to have the same noise patterns as those from the target domain, and the second discriminator enforces the content to be preserved in the generated images. We apply the proposed NAGAN on both optical coherence tomography (OCT) images and ultrasound images. Results show that the method is able to translate the noise style. In addition, we also evaluate our proposed method with segmentation task in OCT and classification task in ultrasound. The experimental results show that the proposed NAGAN improves the analysis outcome.

The estimation of vascular network topology in complex networks is important in understanding the relationship between vascular changes and a wide spectrum of diseases. Automatic classification of the retinal vascular trees into arteries and veins is of direct assistance to the ophthalmologist in terms of diagnosis and treatment of eye disease. However, it is challenging due to their projective ambiguity and subtle changes in appearance, contrast, and geometry in the imaging process. In this paper, we propose a novel method that is capable of making the artery/vein (A/V) distinction in retinal color fundus images based on vascular network topological properties. To this end, we adapt the concept of dominant set clustering and formalize the retinal blood vessel topology estimation and the A/V classification as a pairwise clustering problem. The graph is constructed through image segmentation, skeletonization, and identification of significant nodes. The edge weight is defined as the inverse Euclidean distance between its two end points in the feature space of intensity, orientation, curvature, diameter, and entropy. The reconstructed vascular network is classified into arteries and veins based on their intensity and morphology. The proposed approach has been applied to five public databases, namely INSPIRE, IOSTAR, VICAVR, DRIVE, and WIDE, and achieved high accuracies of 95.1%, 94.2%, 93.8%, 91.1%, and 91.0%, respectively. Furthermore, we have made manual annotations of the blood vessel topologies for INSPIRE, IOSTAR, VICAVR, and DRIVE datasets, and these annotations are released for public access so as to facilitate researchers in the community.

Since age is the most significant risk factor for the development of Alzheimer's disease (AD), it is important to understand the effect of normal ageing on brain network characteristics before we can accurately diagnose the condition based on information derived from resting state electroencephalogram (EEG) recordings, aiming to detect brain network disruption. This article proposes a novel brain functional connectivity imaging method, particularly targeting the contribution of nonlinear dynamics of functional connectivity, on distinguishing participants with AD from healthy controls (HC). We describe a parametric method established upon a Nonlinear Finite Impulse Response model, and a revised orthogonal least squares algorithm used to estimate the linear, nonlinear and combined connectivity between any two EEG channels without fitting a full model. This approach, where linear and non-linear interactions and their spatial distribution and dynamics can be estimated independently, offered us the means to dissect the dynamic brain network disruption in AD from a new perspective and to gain some insight into the dynamic behaviour of brain networks in two age groups (above and below 70) with normal cognitive function. Although linear and stationary connectivity dominates the classification contributions, quantitative results have demonstrated that nonlinear and dynamic connectivity can significantly improve the classification accuracy, barring the group of participants below the age of 70, for resting state EEG recorded during eyes open. The developed approach is generic and can be used as a powerful tool to examine brain network characteristics and disruption in a user friendly and systematic way.

Precise characterization and analysis of corneal nerve fiber tortuosity are of great importance in facilitating examination and diagnosis of many eye-related diseases. In this paper we propose a fully automated method for image-level tortuosity estimation, comprising image enhancement, exponential curvature estimation, and tortuosity level classification. The image enhancement component is based on an extended Retinex model, which not only corrects imbalanced illumination and improves image contrast in an image, but also models noise explicitly to aid removal of imaging noise. Afterwards, we take advantage of exponential curvature estimation in the 3D space of positions and orientations to directly measure curvature based on the enhanced images, rather than relying on the explicit segmentation and skeletonization steps in a conventional pipeline usually with accumulated pre-processing errors. The proposed method has been applied over two corneal nerve microscopy datasets for the estimation of a tortuosity level for each image. The experimental results show that it performs better than several selected state-of-the-art methods. Furthermore, we have performed manual gradings at tortuosity level of four hundred and three corneal nerve microscopic images, and this dataset has been released for public access to facilitate other researchers in the community in carrying out further research on the same and related topics.

In the above article[1], there were two errors in the printed article that the authors want to correct.

The reconstruction and analysis of tree-like topological structures in the biomedical images is crucial for biologists and surgeons to understand biomedical conditions and plan surgical procedures. The underlying tree-structure topology reveals how different curvilinear components are anatomically connected to each other. Existing automated topology reconstruction methods have great difficulty in identifying the connectivity when two or more curvilinear components cross or bifurcate, due to their projection ambiguity, imaging noise and low contrast. In this paper, we propose a novel curvilinear structural similarity measure to guide a dominant-set clustering approach to address this indispensable issue. The novel similarity measure takes into account both intensity and geometric properties in representing the curvilinear structure locally and globally, and group curvilinear objects at crossover points into different connected branches by dominant-set clustering. The proposed method is applicable to different imaging modalities, and quantitative and qualitative results on retinal vessel, plant root, and neuronal network datasets show that our methodology is capable of advancing the current state-of-the-art techniques.

Building transparent knowledge-based systems in the form of accurate and interpretable fuzzy rules is one of the significant applications of fuzzy set theory. The fuzzy connectives, i.e., T -norm/conorm, play the role of connecting fuzzy sets, which are essentially linguistic terms extracted from the knowledge embedded in a given data set. Fuzzy pattern tree is a recently proposed novel machine learning technique, which grows a hierarchical binary tree for each known class utilising conventional T -norms/conorms and aggregation operators. Preaggregation functions are recently proposed in the literature as a type of generalised aggregation functions, which have achieved successes in a number of applications. This paper proposes a preaggregation-based approach with application to the construction of fuzzy pattern tree. An experimental study is done to explore the performance of the fuzzy pattern tree where preaggregation functions are employed in comparison to that where conventional aggregation operators are utilised. Experimental results demonstrate that the performance of fuzzy pattern tree incorporated with the preaggregation function generated by Nilpotent minimum T -norm outperforms those with alternative preaggregation functions and the commonly used ordered weighted averaging operators.

Medical image segmentation is an important step in medical image analysis. With the rapid development of a convolutional neural network in image processing, deep learning has been used for medical image segmentation, such as optic disc segmentation, blood vessel detection, lung segmentation, cell segmentation, and so on. Previously, U-net based approaches have been proposed. However, the consecutive pooling and strided convolutional operations led to the loss of some spatial information. In this paper, we propose a context encoder network (CE-Net) to capture more high-level information and preserve spatial information for 2D medical image segmentation. CE-Net mainly contains three major components: a feature encoder module, a context extractor, and a feature decoder module. We use the pretrained ResNet block as the fixed feature extractor. The context extractor module is formed by a newly proposed dense atrous convolution block and a residual multi-kernel pooling block. We applied the proposed CE-Net to different 2D medical image segmentation tasks. Comprehensive results show that the proposed method outperforms the original U-Net method and other state-of-the-art methods for optic disc segmentation, vessel detection, lung segmentation, cell contour segmentation, and retinal optical coherence tomography layer segmentation.

Glaucoma is one of the leading cause of blindness. Although there is still no cure, early detection can prevent serious vision loss. Therefore automated glaucoma detection/classification is an important issue. In the past decade, segmentation based approach such as those based on cup-to-disc-ratio are popular, but single indicator limit its performance. Recently, convolutional neural network based image classification approaches that can use more image cues achieve good performance. In this paper, we propose a new glaucoma classification by combining multiple features extracted by different convolutional neural networks. Its effectiveness is clearly demonstrated on the publicly available Origa [1] dataset. It achieves an area under the receiver operating characteristic curve of 0.8483, which better than the 0.838 given by on manual marked cup-to-disc-ratio. To our knowledge, it is the first approach surpass human in glaucoma classification.

Retinal fundus photographs have been used in the diagnosis of many ocular diseases such as glaucoma, pathological myopia, age-related macular degeneration, and diabetic retinopathy. With the development of computer science, computer aided diagnosis has been developed to process and analyze the retinal images automatically. One of the challenges in the analysis is that the quality of the retinal image is often degraded. For example, a cataract in human lens will attenuate the retinal image, just as a cloudy camera lens which reduces the quality of a photograph. It often obscures the details in the retinal images and posts challenges in retinal image processing and analyzing tasks. In this paper, we approximate the degradation of the retinal images as a combination of human-lens attenuation and scattering. A novel structure-preserving guided retinal image filtering (SGRIF) is then proposed to restore images based on the attenuation and scattering model. The proposed SGRIF consists of a step of global structure transferring and a step of global edge-preserving smoothing. Our results show that the proposed SGRIF method is able to improve the contrast of retinal images, measured by histogram flatness measure, histogram spread, and variability of local luminosity. In addition, we further explored the benefits of SGRIF for subsequent retinal image processing and analyzing tasks. In the two applications of deep learning-based optic cup segmentation and sparse learning-based cup-to-disk ratio (CDR) computation, our results show that we are able to achieve more accurate optic cup segmentation and CDR measurements from images processed by SGRIF.

Glaucoma is a chronic eye disease that leads to irreversible vision loss. The cup to disc ratio (CDR) plays an important role in the screening and diagnosis of glaucoma. Thus, the accurate and automatic segmentation of optic disc (OD) and optic cup (OC) from fundus images is a fundamental task. Most existing methods segment them separately, and rely on hand-crafted visual feature from fundus images. In this paper, we propose a deep learning architecture, named M-Net, which solves the OD and OC segmentation jointly in a one-stage multi-label system. The proposed M-Net mainly consists of multi-scale input layer, U-shape convolutional network, side-output layer, and multi-label loss function. The multi-scale input layer constructs an image pyramid to achieve multiple level receptive field sizes. The U-shape convolutional network is employed as the main body network structure to learn the rich hierarchical representation, while the side-output layer acts as an early classifier that produces a companion local prediction map for different scale layers. Finally, a multi-label loss function is proposed to generate the final segmentation map. For improving the segmentation performance further, we also introduce the polar transformation, which provides the representation of the original image in the polar coordinate system. The experiments show that our M-Net system achieves state-of-the-art OD and OC segmentation result on ORIGA data set. Simultaneously, the proposed method also obtains the satisfactory glaucoma screening performances with calculated CDR value on both ORIGA and SCES datasets.

Glaucoma is a chronic eye disease that leads to irreversible vision loss. Most of the existing automatic screening methods first segment the main structure and subsequently calculate the clinical measurement for the detection and screening of glaucoma. However, these measurement-based methods rely heavily on the segmentation accuracy and ignore various visual features. In this paper, we introduce a deep learning technique to gain additional image-relevant information and screen glaucoma from the fundus image directly. Specifically, a novel disc-aware ensemble network for automatic glaucoma screening is proposed, which integrates the deep hierarchical context of the global fundus image and the local optic disc region. Four deep streams on different levels and modules are, respectively, considered as global image stream, segmentation-guided network, local disc region stream, and disc polar transformation stream. Finally, the output probabilities of different streams are fused as the final screening result. The experiments on two glaucoma data sets (SCES and new SINDI data sets) show that our method outperforms other state-of-the-art algorithms.

Automated detection of vascular structures is of great importance in understanding the mechanism, diagnosis, and treatment of many vascular pathologies. However, automatic vascular detection continues to be an open issue because of difficulties posed by multiple factors, such as poor contrast, inhomogeneous backgrounds, anatomical variations, and the presence of noise during image acquisition. In this paper, we propose a novel 2-D/3-D symmetry filter to tackle these challenging issues for enhancing vessels from different imaging modalities. The proposed filter not only considers local phase features by using a quadrature filter to distinguish between lines and edges, but also uses the weighted geometric mean of the blurred and shifted responses of the quadrature filter, which allows more tolerance of vessels with irregular appearance. As a result, this filter shows a strong response to the vascular features under typical imaging conditions. Results based on eight publicly available datasets (six 2-D data sets, one 3-D data set, and one 3-D synthetic data set) demonstrate its superior performance to other state-of-the-art methods.

Angle-closure glaucoma is a major cause of irreversible visual impairment and can be identified by measuring the anterior chamber angle (ACA) of the eye. The ACA can be viewed clearly through anterior segment optical coherence tomography (AS-OCT), but the imaging characteristics and the shapes and locations of major ocular structures can vary significantly among different AS-OCT modalities, thus complicating image analysis. To address this problem, we propose a data-driven approach for automatic AS-OCT structure segmentation, measurement, and screening. Our technique first estimates initial markers in the eye through label transfer from a hand-labeled exemplar data set, whose images are collected over different patients and AS-OCT modalities. These initial markers are then refined by using a graph-based smoothing method that is guided by AS-OCT structural information. These markers facilitate segmentation of major clinical structures, which are used to recover standard clinical parameters. These parameters can be used not only to support clinicians in making anatomical assessments, but also to serve as features for detecting anterior angle closure in automatic glaucoma screening algorithms. Experiments on Visante AS-OCT and Cirrus high-definition-OCT data sets demonstrate the effectiveness of our approach.

We present a one-step approach based on low-rank tensor recovery for axial alignment in 360-degree anterior chamber optical coherence tomography. Achieving translational alignment and rotation correction of cross-sections simultaneously, this technique obtains a better anterior segment topographical representation and improves quantitative measurement accuracy and reproducibility of disease related parameters. Through its use of global information, the proposed method is more robust compared to using only individual or paired slices, and less sensitive to noise and motion artifacts. In angle closure analysis on 30 patient eyes, the preliminary results indicate that the proposed axial alignment method can not only facilitate manual qualitative analysis with more distinct landmark representation and much less human labor, but also can improve the accuracy of automatic quantitative assessment by 2.9 %, which demonstrates that the proposed approach is promising for a wide range of clinical applications. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Cataracts are the leading cause of visual impairment and blindness worldwide. Cataract grading, i.e. assessing the presence and severity of cataracts, is essential for diagnosis and progression monitoring. We present in this work an automatic method for predicting cataract grades from slit-lamp lens images. Different from existing techniques which normally formulate cataract grading as a regression problem, we solve it through reconstruction-based classification, which has been shown to yield higher performance when the available training data is densely distributed within the feature space. To heighten the effectiveness of this reconstruction-based approach, we introduce a new semantic feature representation that facilitates alignment of test and reference images, and include locality constraints on the linear reconstruction to reduce the influence of less relevant reference samples. In experiments on the large ACHIKO-NC database comprised of 5378 images, our system outperforms the state-of-the-art regression methods over a range of evaluation metrics. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Optical coherence tomography (OCT) is a micrometer-scale, cross-sectional imaging modality for biological tissue. It has been widely used for retinal imaging in ophthalmology. Speckle noise is problematic in OCT. A raw OCT image/volume usually has very poor image quality due to speckle noise, which often obscures the retinal structures. Overlapping scan is often used for speckle reduction in a 2D line-scan. However, it leads to an increase of the data acquisition time. Therefore, it is unpractical in 3D scan as it requires a much longer data acquisition time. In this paper, we propose a new method for speckle reduction in 3D OCT. The proposed method models each A -scan as the sum of underlying clean A -scan and noise. Based on the assumption that neighboring A -scans are highly similar in the retina, the method reconstructs each A -scan from its neighboring scans. In the method, the neighboring A -scans are aligned/registered to the A -scan to be reconstructed and form a matrix together. Then low rank matrix completion using bilateral random projection is utilized to iteratively estimate the noise and recover the underlying clean A -scan. The proposed method is evaluated through the mean square error, peak signal to noise ratio and the mean structure similarity index using high quality line-scan images as reference. Experimental results show that the proposed method performs better than other methods. In addition, the subsequent retinal layer segmentation also shows that the proposed method makes the automatic retinal layer segmentation more accurate. The technology can be embedded into current OCT machines to enhance the image quality for visualization and subsequent analysis such as retinal layer segmentation.

We present an object-based co-segmentation method that takes advantage of depth data and is able to correctly handle noisy images in which the common foreground object is missing. With RGBD images, our method utilizes the depth channel to enhance identification of similar foreground objects via a proposed RGBD co-saliency map, as well as to improve detection of object-like regions and provide depth-based local features for region comparison. To accurately deal with noisy images where the common object appears more than or less than once, we formulate co-segmentation in a fully-connected graph structure together with mutual exclusion (mutex) constraints that prevent improper solutions. Experiments show that this object-based RGBD co-segmentation with mutex constraints outperforms related techniques on an RGBD co-segmentation dataset, while effectively processing noisy images. Moreover, we show that this method also provides performance comparable to state-of-the-art RGB co-segmentation techniques on regular RGB images with depth maps estimated from them.

Existing feature descriptor-based methods on retinal image registration are mainly based on scale-invariant feature transform (SIFT) or partial intensity invariant feature descriptor (PIIFD). While these descriptors are often being exploited, they do not work very well upon unhealthy multimodal images with severe diseases. Additionally, the descriptors demand high dimensionality to adequately represent the features of interest. The higher the dimensionality, the greater the consumption of resources (e.g. memory space). To this end, this paper introduces a novel registration algorithm coined low-dimensional step pattern analysis (LoSPA), tailored to achieve low dimensionality while providing sufficient distinctiveness to effectively align unhealthy multimodal image pairs. The algorithm locates hypotheses of robust corner features based on connecting edges from the edge maps, mainly formed by vascular junctions. This method is insensitive to intensity changes, and produces uniformly distributed features and high repeatability across the image domain. The algorithm continues with describing the corner features in a rotation invariant manner using step patterns. These customized step patterns are robust to non-linear intensity changes, which are well-suited for multimodal retinal image registration. Apart from its low dimensionality, the LoSPA algorithm achieves about two-fold higher success rate in multimodal registration on the dataset of severe retinal diseases when compared to the top score among state-of-the-art algorithms.

This paper presents a novel retina biometric scheme that performs person verification based on passing 2 stages: robust feature points matching and edge dissimilarity measure. Our approach differs from those in the literature as we propose the use of edges and edge dissimilarity measure for retina verification. Our first-stage matching/authentication utilizes robust feature points' matching to determine tentatively whether there is a “match” and if so, performs image registration between the test and template retina image. The robust feature points' matching is achieved in 2 steps: graph-based feature points' matching followed by pruning of wrongly matched feature points using a Least-Median-Squares estimator that enforces an affine transformation geometric constraint. To compute edge dissimilarity measure in our second-stage matching/authentication, we propose the “robustified Hausdorff distance”. We show that our proposed approach outperforms two of the state-of-the-art approaches when tested on the same dataset.

Age-related Macular Degeneration (AMD) is the third leading cause of blindness. Its prevalence is increasing in these years for the coming of "aging time". Early detection and grading can prohibit it from becoming severe and protect vision. The appearance of drusen is an important indicator for AMD thus automatic drusen detection and segmentation have attracted much research attention in the past years. In this paper, we propose a novel drusen segmentation method by using Growcut. This method first detects the local maximum and minimum points. The maximum points, which are potential drusen, are then classified as drusen or non-drusen. The drusen points will be used as foreground labels while the non-drusen points together with the minima will be used as background labels. These labels are fed into Growcut to obtain the drusen boundaries. The method is tested on a manually labeled dataset with 96 images containing drusen. The experimental results verify the effectiveness of the method.

BACKGROUND: Computer-aided diagnosis for screening utilizes computer-based analytical methodologies to process patient information. Glaucoma is the leading irreversible cause of blindness. Due to the lack of an effective and standard screening practice, more than 50% of the cases are undiagnosed, which prevents the early treatment of the disease. OBJECTIVE: To design an automatic glaucoma diagnosis architecture automatic glaucoma diagnosis through medical imaging informatics (AGLAIA-MII) that combines patient personal data, medical retinal fundus image, and patient's genome information for screening. MATERIALS AND METHODS: 2258 cases from a population study were used to evaluate the screening software. These cases were attributed with patient personal data, retinal images and quality controlled genome data. Utilizing the multiple kernel learning-based classifier, AGLAIA-MII, combined patient personal data, major image features, and important genome single nucleotide polymorphism (SNP) features. RESULTS AND DISCUSSION: Receiver operating characteristic curves were plotted to compare AGLAIA-MII's performance with classifiers using patient personal data, images, and genome SNP separately. AGLAIA-MII was able to achieve an area under curve value of 0.866, better than 0.551, 0.722 and 0.810 by the individual personal data, image and genome information components, respectively. AGLAIA-MII also demonstrated a substantial improvement over the current glaucoma screening approach based on intraocular pressure. CONCLUSIONS: AGLAIA-MII demonstrates for the first time the capability of integrating patients' personal data, medical retinal image and genome information for automatic glaucoma diagnosis and screening in a large dataset from a population study. It paves the way for a holistic approach for automatic objective glaucoma diagnosis and screening.

Glaucoma is a chronic eye disease that leads to vision loss. As it cannot be cured, detecting the disease in time is important. Current tests using intraocular pressure (IOP) are not sensitive enough for population based glaucoma screening. Optic nerve head assessment in retinal fundus images is both more promising and superior. This paper proposes optic disc and optic cup segmentation using superpixel classification for glaucoma screening. In optic disc segmentation, histograms, and center surround statistics are used to classify each superpixel as disc or non-disc. A self-assessment reliability score is computed to evaluate the quality of the automated optic disc segmentation. For optic cup segmentation, in addition to the histograms and center surround statistics, the location information is also included into the feature space to boost the performance. The proposed segmentation methods have been evaluated in a database of 650 images with optic disc and optic cup boundaries manually marked by trained professionals. Experimental results show an average overlapping error of 9.5% and 24.1% in optic disc and optic cup segmentation, respectively. The results also show an increase in overlapping error as the reliability score is reduced, which justifies the effectiveness of the self-assessment. The segmented optic disc and optic cup are then used to compute the cup to disc ratio for glaucoma screening. Our proposed method achieves areas under curve of 0.800 and 0.822 in two data sets, which is higher than other methods. The methods can be used for segmentation and glaucoma screening. The self-assessment will be used as an indicator of cases with large errors and enhance the clinical deployment of the automatic segmentation and screening.

The vertical Cup-to-Disc Ratio (CDR) is an important indicator in the diagnosis of glaucoma. Automatic segmentation of the optic disc (OD) and optic cup is crucial towards a good computer-aided diagnosis (CAD) system. This paper presents a statistical model-based method for the segmentation of the optic disc and optic cup from digital color fundus images. The method combines knowledge-based Circular Hough Transform and a novel optimal channel selection for segmentation of the OD. Moreover, we extended the method to optic cup segmentation, which is a more challenging task. The system was tested on a dataset of 325 images. The average Dice coefficient for the disc and cup segmentation is 0.92 and 0.81 respectively, which improves significantly over existing methods. The proposed method has a mean absolute CDR error of 0.10, which outperforms existing methods. The results are promising and thus demonstrate a good potential for this method to be used in a mass screening CAD system.

Age-related macular degeneration (AMD) is a leading cause of vision loss. The presence of drusen are often associated to AMD. Drusen are tiny yellowish-white extracellular buildup present around the macular region of the retina. Clinically, ophthalmologists examine the area around the macula to determine the presence and severity of drusen. However, manual identification and recognition of drusen is subjective, time consuming and expensive. To reduce manual workload and facilitate large-scale early AMD screening, it is essential to detect drusen automatically. In this paper, we propose to use biologically inspired features (BIF) for the purpose of AMD detection. The optic disc and macula are detected to determine a focal region around macula for feature extraction. The extracted features are then classified using support vector machines (SVM). Our experimental results, tested on 350 images, demonstrate that the biologically inspired features from the focal region is effective for drusen detection with a sensitivity of 86.3% and specificity of 91.9%. The results of our proposed approach can be used to reduce workload of ophthalmologists and diagnosis cost.

Peripapillary atrophy (PPA) is an atrophy of pre-existing retina tissue. Because of its association with eye diseases such as myopia and glaucoma, PPA is an important indicator for diagnosis of these diseases. Experienced ophthalmologists are able to determine the presence of PPA using visual information from the retinal images. However, it is tedious, time consuming and subjective to examine all images especially in a screening program. This paper presents biologically inspired feature (BIF) for the automatic detection of PPA. BIF mimics the process of cortex for visual perception. In the proposed method, a focal region is segmented from the retinal image and the BIF is extracted. As BIF is an intrinsically low dimensional feature embedded in a high dimensional space, it is not suitable to measure the similarity between two BIFs directly based on the Euclidean distance. Therefore, it is necessary to obtain a suitable mapping to reduce the dimensionality. In this paper, we explore sparse transfer learning to transfer the label information from ophthalmologists to the sample distribution knowledge contained in all samples. Selective pair-wise discriminant analysis is used to define two strategies of sparse transfer learning: negative and positive sparse transfer learning. Experimental results show that negative sparse transfer learning is superior to the positive one for this task. The proposed BIF based approach achieves an accuracy of more than 90% in detecting PPA, much better than previous methods. It can be used to save the workload of ophthalmologists and thus reduce the diagnosis costs.

In clinical diagnosis, a grade indicating the severity of nuclear cataract is often manually assigned by a trained ophthalmologist to a patient after comparing the lens' opacity severity in his/her slit-lamp images with a set of standard photos. This grading scheme is often subjective and time-consuming. In this paper, a novel computer-aided diagnosis method via ranking is proposed to facilitate nuclear cataract grading following conventional clinical decision-making process. The grade of nuclear cataract in a slit-lamp image is predicted using its neighboring labeled images in a ranked image list, which is achieved using a learned ranking function. This ranking function is learned via direct optimization on a newly proposed approximation to a ranking evaluation measure. Our proposed method has been evaluated by a large dataset composed of 1000 different cases, which are collected from an ongoing clinical population-based study. Both experimental results and comparison with several existing methods demonstrate the benefit of grading via ranking by our proposed method.

Blastocystis is a unicellular but polymorphic protozoan parasite causing digestive diseases in humans. Autophagy, a self-degradation process, is only recently found in Blastocystis. Identifying and enumerating autophagic Blastocystis cells using fluorescent microscopy are important in biology. Doing this manually is laborious and error-prone. This paper proposes image analysis techniques to automate the process. The difficulties are poor image quality and large variations in illumination and cell morphology. We divide the cells into several sub-classes of different morphology. Support vector machines are used to learn domain knowledge and classify the cells. Validation experiments on separate data sets show reliable performance for manually segmented cells with sensitivity 82.2% and specificity 86.7%. For automatically segmented cells, the sensitivity is the same. However, the specificity drops down to 68.4%. To our knowledge, this is the first attempt in automatic processing these images.

This paper presents a photometric restoration technique that automatically corrects shading within retinal images taken with a fundus camera. The proposed technique is based on the observation that the background of retinal images usually shows flat reflectance variations due to its high similarity in color and texture. It estimates shading through an iterative polynomial interpolation procedure that first estimates a shading image through a horizontal interpolation process and then improves the shading estimation by a vertical interpolation process. Once the shading image is estimated, a reflectance image can accordingly be determined based on the luminance of the retina image under study. Experiments on 161 retinal images of different qualities show promising results.

The construction of probabilistic liver atlases has received little attention in the past. Existing methods are based on landmarks and are sensitive to their choices and placements. We propose an iterative landmark-free method based on dense volumes to construct linear unbiased diffeomorphic probabilistic atlases from liver CT images. The linear averaging of the transformed images is set as the common target space followed by pairwise diffeomorphic registrations to warp all images to the target using a recent-proposed efficient deformation approach during each iteration cycle. Iterative pairwise registrations are directly used to handle possible large deformations without the need for an extra step to remove global deformations such as the use of affine transformations in traditional methods. Compared with those approaches estimating the unbiased atlas and the transformations groupwise simultaneously, the current method is more efficient. The efficiency and the convergence of our method have been demonstrated experimentally by validation using 25 CT liver sets.

Computer aided analysis of medical images, a unique type of non-text media, can facilitate clinical diagnosis. As an example, an automatic opacity detection approach is proposed in this paper to grade cortical cataract more objectively. The automatic pupil detection is performed by detecting the strongest edges on the convex hull and ellipse fitting using nonlinear least square method. The cortical opacity is detected by radial edge detection and post-processing. The automatic grades are assigned following Wisconsin cataract grading protocol. The accuracy of pupil detection is 98.2%. The mean error of opacity area detection is 7 percent compared with the result of human grader. And 86.3% accurate grades of cortical cataract are achieved. This is the first time that the spoke-like feature is utilized in the automatic detection of cortical cataract to separate from other opacity types. The encouraging results show that it is probable to apply the proposed approach to clinical diagnosis later.

Automatic classification of working areas in peripheral blood smears can provide objective and reproducible quality control for the evaluation of smears and smear maker devices. However, it has drawn little research attention. In this paper we study this topic using image analysis and statistical pattern recognition methods. We employ generic features without requiring the extraction of individual cells. Two new spatial distribution features across scales are defined and utilized to classify working areas. We demonstrate that the only feature and method proposed in a similar work by others is insufficient to characterize the goodness of working areas, particularly the cell distribution. However, by utilizing it together with the features developed in this paper, we can achieve much better results. Our method has been tested on about 150 labeled images acquired from three malaria-infected Giemsa-stained blood smears using an oil immersion 100x objective lens.

This paper describes a new hybrid approach set-based hybrid approach (SHA) for magnetic resonance (MR) image segmentation by integrating two existing techniques, region-grow and threshold level set. To evaluate the proposed approach in performing real world image segmentation task, instead of using well-taken MR-images, we use real-life images collected in a hospital. Comparison of the performance between the two individual techniques and the new hybrid technique demonstrates the effectiveness of the latter

S-AdaBoost is a new variant of AdaBoost and is more effective than the conventional AdaBoost in handling outliers in pattern detection and classification in real world complex environment. Utilizing the divide and conquer principle, S-AdaBoost divides the input space into a few sub-spaces and uses dedicated classifiers to classify patterns in the sub-spaces. The final classification result is the combination of the outputs of the dedicated classifiers. S-AdaBoost system is made up of an AdaBoost divider, an AdaBoost classifier, a dedicated classifier for outliers, and a non-linear combiner. In addition to presenting face detection test results in a complex airport environment, we have also conducted experiments on a number of benchmark databases to test the algorithm. The experiment results clearly show S-AdaBoost's effectiveness in pattern detection and classification.