Knowledge Graph (KG)-based Retrieval-Augmented Generation (RAG) shifts the contents of retrieval from narrative text to a relational knowledge network, empowering large language models (LLMs) to harness structured relationships between entities. However, conventional KG-RAG approaches are resource-intensive, requiring either query decomposition with multiple LLM rounds or parameterized static knowledge injection to update the model. Although subgraph reasoning aims to address these issues, most current methods are based on heuristic shortest path and multi-hop graph traversal algorithms. The retrieved subgraphs suffer from incompleteness and semantic drift, and neglect the interaction between subgraph and LLMs in terms of fine-grained structural semantics. We propose a dual-constraint subgraph optimization for KG-RAG (DCTR). It improves subgraph retrieval and generates high-quality subgraphs with structural integrity and information salience for LLMs. Specifically, it formulates subgraph generation as a two-stage graph-theoretic constrained optimization problem to create compact and complete pseudo-labels. Since these pseudo-labels are discrete, a smooth approximation is employed to convert them into a differentiable representation, thereby optimizing the retriever to highlight key information while extracting subgraphs. On two benchmark datasets, DCTR significantly enhances subgraph quality, achieving state-of-the-art performance in LLM reasoning.

Multi-Hop Question Answering (MHQA) requires step-by-step reasoning across multiple pieces of information to answer complex questions. The cache-aided Retrieval-Augmented Generation (RAG) can accelerate the process of external knowledge retrieval at each reasoning step for MHQA. However, existing methods focus on the internal structure and ignore the misalignment between the queries’ arrival order and cache hit order. To tackle this, we propose Mnemosyne, a cache hit order fitting method designed to accelerate the RAG progress for MHQA. Specifically, our cache-aware order fitting strategy adjusts the order of queries arrival via graph reordering to better align with the cache hit order, thereby reducing the likelihood of failed or unproductive retrieval attempts. The multi-granularity caching storage mechanism is designed to loosen the strict hit condition to multiple similar semantic matching modes, facilitating that relevant documents can still be retrieved. Experiments conducted on four multi-hop QA datasets demonstrate that Mnemosyne effectively reduces retrieval latency while enhancing task answer F1 score, achieving a superior trade-off between efficiency and effectiveness.

Cone-beam computed tomography (CBCT) is widely used for dento-maxillofacial diagnostics and treatment planning, and comprehensive multi-structure segmentation remains time-consuming, limiting large-scale, reproducible research. In this article, we present ToothFairy2, a MICCAI 2024 challenge on multi-structure segmentation in maxillofacial CBCT. The accompanying dataset comprises 530 CBCT volumes (480 public training, 50 hidden test) with expert 3D annotations of 42 classes, including maxilla, mandible, crowns, bridges, implants, inferior alveolar canals, maxillary sinuses, pharynx, and teeth labeled according to the International Tooth Numbering System (FDI). 26 international teams participated in ToothFairy2, and their methods were run and evaluated for voxel-wise multi-class segmentation using a standardized protocol. This report extends the evaluation of teeth to also investigate the current capabilities of tooth detection and FDI numbering. Furthermore, ranking stability was analyzed to assess the robustness of the final challenge outcome. Overall, challenge participants achieved consistently high performance for large, high-contrast structures such as jawbones, pharynx, and most teeth, while maxillary sinuses, dental restorations, and fine structures remain challenging due to class imbalance and metal artifacts. Analysis of tooth-related metrics further revealed that assigning correct FDI numbers was more challenging than delineating individual teeth. By releasing CBCT data, 3D annotations, baseline models, and evaluation code, ToothFairy2 establishes a long-term benchmark to drive the development of automated methods for robust, clinically meaningful multi-structure segmentation in maxillofacial CBCT.

Accurate delineation of pancreatic tumors on Magnetic Resonance Imaging (MRI) is important for diagnosis, radiotherapy treatment planning, and outcome assessment, but remains challenging due to complex anatomy and subtle tumor appearance. In routine practice, tumor contours on MRI are produced manually, which is time-consuming and subject to inter-observer variability. Radiotherapy on MRI-Linear Accelerator (MRI-Linac) systems further requires fast and consistent Gross Tumor Volume (GTV) contours for online adaptation, yet most public pancreas tumor segmentation benchmarks focus on Computed Tomography (CT). The Pancreatic Tumor Segmentation in Therapeutic and Diagnostic MRI (PANTHER) challenge addresses this gap by benchmarking automatic pancreatic tumor segmentation on MRI. The dataset includes contrast-enhanced T1-weighted diagnostic MRI and T2-weighted MRI-Linac scans with expert pancreas and tumor annotations, organized into two tasks: (1) tumor segmentation on diagnostic MRI and (2) tumor segmentation on MRI-Linac images. Performance was evaluated using overlap metrics, distance-based metrics, and tumor volume error. The challenge attracted 285 registered participants, with 12 and 9 final submissions for Tasks 1 and 2, respectively. On diagnostic MRI, top methods achieved performance close to inter-reader agreement. Multi-reader analysis suggested that models often reproduced the contouring style of the training annotator, highlighting the importance of annotation quality and consensus. In contrast, performance on MRI-Linac images was lower and more heterogeneous, including cases of complete localization failure. PANTHER provides the first public benchmark for pancreatic tumor segmentation on MRI, showing that clinically useful automation is feasible on diagnostic MRI, while robust MRI-Linac GTV segmentation remains an open challenge.

Deep learning methods have been proposed for tooth segmentation on high-resolution intra-oral scans (IOS) that plays a crucial role in clinical dental practice. However, they generally segment teeth in a low-resolution data with a fixed receptive field and generate final segmentation by up-sampling interpolation, and neglect teeth's morphology priors: their similar dental arch structures and significantly different curvatures in different parts of each tooth. They thus lack adaptability to different parts of each tooth, and show less accurate segmentation of boundary points between teeth and gums due to the up-sampling computation. Further, cluttered poses of IOS limit their generalization and usability of teeth location and geometric information. To address these limitations, a morphology prior enhanced teeth segmentation framework is proposed in this paper. Firstly, a robust preprocessing is introduced to align poses of different IOS by computing their dental arch orientations, thereby improving segmentation generalization and usability of IOS geometric information. Secondly, a decomposition-merging strategy is designed to avoid the up-sampling limitation, which decomposes an IOS into multiple low-resolution data and merges their segmentation outcomes into a high-resolution result. Thirdly, an innovative module integrating semantic and geometric features is proposed to adaptively select deformable receptive fields. It geometrically samples within a variable probability space to construct receptive fields with varied graph relationships for different points, facilitating adaptive segmentation of different parts of each tooth. Experimental results on 6238 IOS from four centers demonstrate that our method significantly outperforms 11 state-of-the-art methods, achieving a 6.93% enhancement for cross-center testing.

Temporal knowledge graph (TKG) reasoning that infers future missing facts is an essential and challenging task. Predicting future events typically relies on closely related historical facts, yielding more accurate results for repetitive or periodic events. However, for future events with sparse historical interactions, the effectiveness of this method, which focuses on leveraging high-frequency historical information, diminishes. Recently, the capabilities of diffusion models in image generation have opened new opportunities for TKG reasoning. Therefore, we propose a graph node diffusion model with dual-domain periodic contrastive learning (DPCL-Diff). Graph node diffusion model (GNDiff) introduces noise into sparsely related events to simulate new events, generating high-quality data that better conforms to the actual distribution. This generative mechanism significantly enhances the model's ability to reason about new events. Additionally, the dual-domain periodic contrastive learning (DPCL) maps periodic and non-periodic event entities to Poincaré and Euclidean spaces, leveraging their characteristics to distinguish similar periodic events effectively. Experimental results on four public datasets demonstrate that DPCL-Diff significantly outperforms state-of-the-art TKG models in event prediction, demonstrating our approach's effectiveness. This study also investigates the combined effectiveness of GNDiff and DPCL in TKG tasks.

Emotion recognition in conversation (ERC) aims to identify the emotion expressed in each utterance within a multi-turn dialogue. In recent years, the widespread adoption of language models (LM) has spurred the development of various prompting paradigms as effective adaptation strategies, aligning LM training objectives with the specific requirements of ERC. However, while the hard prompt-based paradigm offers high human interpretability, it suffers from limited task adaptability due to its discrete nature. In contrast, the soft prompt-based paradigm sacrifices interpretability in favor of improved adaptability by optimizing continuous embedding vectors. Both paradigms generally adopt an invariant prompt across utterances, which restricts their ability to model contextual diversity and results in suboptimal adaptability of the instance. To address these issues, we propose a comprehensive prompting strategy that balances interpretability and adaptability, consisting of two components: reinforced prompt exploration for hard prompts and feature integration for soft prompts. In reinforced prompt exploration, a policy network is trained via reinforcement learning to explore the discrete prompt space under cold-start conditions, efficiently optimizing hard prompts and improving task-level adaptability while preserving interpretability. In feature integration for soft prompts, we incorporate rich semantic features to form contextually relevant soft prompts, assigning each utterance a distinct offset subspace to improve instance-level adaptability. Experiments on three datasets demonstrate that our prompting strategy achieves state-of-the-art performance in ERC.

Emotion Recognition in Conversation (ERC) is a challenging task in conversational AI systems. However, current ERC methods that rely on language models (LM) often use fixed prompts, which lack dynamic adaptability and fail to adequately incorporate external knowledge influencing conversational emotions. To address these limitations, this paper proposes the Dual-Intervention Prompt Framework for emotion recognition in conversation (DIPF). DIPF introduces two types of prompts: explicit prompts and implicit prompts. Explicit prompts leverage external knowledge via knowledge graphs, allowing the model to better simulate conversational flow and emotional dynamics. Implicit prompts are integrated into the hidden layers of the LM, becoming part of the model’s representation and participating in the gradient descent process. Additionally, DIPF manipulates the model representations corresponding to implicit prompts using a low-rank projection matrix, guiding the model’s behavior during inference and improving the optimization of implicit prompts. Extensive experiments on three benchmark datasets demonstrate that DIPF achieves relative F1 score improvements of 1.27% on IEMOCAP, 2.26% on MELD, and 0.72% on EmoryNLP over the strongest baseline (EACL), with an average relative improvement of 1.52%. These results confirm the effectiveness of DIPF in enhancing emotional understanding through dynamic and knowledge-aware prompting strategies.

In recent years, several algorithms have been developed for the segmentation of the Inferior Alveolar Canal (IAC) in Cone-Beam Computed Tomography (CBCT) scans. However, the availability of public datasets in this domain is limited, resulting in a lack of comparative evaluation studies on a common benchmark. To address this scientific gap and encourage deep learning research in the field, the ToothFairy challenge was organized within the MICCAI 2023 conference. In this context, a public dataset was released to also serve as a benchmark for future research. The dataset comprises 443 CBCT scans, with voxel-level annotations of the IAC available for 153 of them, making it the largest publicly available dataset of its kind. The participants of the challenge were tasked with developing an algorithm to accurately identify the IAC using the 2D and 3D-annotated scans. This paper presents the details of the challenge and the contributions made by the most promising methods proposed by the participants. It represents the first comprehensive comparative evaluation of IAC segmentation methods on a common benchmark dataset, providing insights into the current state-of-the-art algorithms and outlining future research directions. Furthermore, to ensure reproducibility and promote future developments, an open-source repository that collects the implementations of the best submissions was released.

Pediatric teeth exhibit significant changes in type and spatial distribution across different age groups. This variation makes pediatric teeth segmentation from cone-beam computed tomography (CBCT) more challenging than that in adult teeth. Existing methods mainly focus on adult teeth segmentation, which however cannot be adapted to spatial distribution of pediatric teeth with individual changes (SDPTIC) in different children, resulting in limited accuracy for segmenting pediatric teeth. Therefore, we introduce a novel topology structure-guided graph convolutional network (TSG-GCN) to generate dynamic graph representation of SDPTIC for improved pediatric teeth segmentation. Specifically, this network combines a 3D GCN-based decoder for teeth segmentation and a 2D decoder for dynamic adjacency matrix learning (DAML) to capture SDPTIC information for individual graph representation. 3D teeth labels are transformed into specially-designed 2D projection labels, which is accomplished by first decoupling 3D teeth labels into class-wise volumes for different teeth via one-hot encoding and then projecting them to generate instance-wise 2D projections. With such 2D labels, DAML can be trained to adaptively describe SDPTIC from CBCT with dynamic adjacency matrix, which is then incorporated into GCN for improving segmentation. To ensure inter-task consistency at the adjacency matrix level between the two decoders, a novel loss function is designed. It can address the issue with inconsistent prediction and unstable TSG-GCN convergence due to two heterogeneous decoders. The TSG-GCN approach is finally validated with both public and multi-center datasets. Experimental results demonstrate its effectiveness for pediatric teeth segmentation, with significant improvement over seven state-of-the-art methods.

Delineating 3D blood vessels of various anatomical structures is essential for clinical diagnosis and treatment, however, is challenging due to complex structure variations and varied imaging conditions. Although recent supervised deep learning models have demonstrated their superior capacity in automatic 3D vessel segmentation, the reliance on expensive 3D manual annotations and limited capacity for annotation reuse among different vascular structures hinder their clinical applications. To avoid the repetitive and costly annotating process for each vascular structure and make full use of existing annotations, this paper proposes a novel 3D shape-guided local discrimination (3D-SLD) model for 3D vascular segmentation under limited guidance from public 2D vessel annotations. The primary hypothesis is that 3D vessels are composed of semantically similar voxels and often exhibit tree-shaped morphology. Accordingly, the 3D region discrimination loss is firstly proposed to learn the discriminative representation measuring voxel-wise similarities and cluster semantically consistent voxels to form the candidate 3D vascular segmentation in unlabeled images. Secondly, the shape distribution from existing 2D structure-agnostic vessel annotations is introduced to guide the 3D vessels with the tree-shaped morphology by the adversarial shape constraint loss. Thirdly, to enhance training stability and prediction credibility, the highlighting-reviewing-summarizing (HRS) mechanism is proposed. This mechanism involves summarizing historical models to maintain temporal consistency and identifying credible pseudo labels as reliable supervision signals. Only guided by public 2D coronary artery annotations, our method achieves results comparable to SOTA barely-supervised methods in 3D cerebrovascular segmentation, and the best DSC in 3D hepatic vessel segmentation, demonstrating the effectiveness of our method.

Fetal Magnetic Resonance Imaging (MRI) is challenged by fetal movements and maternal breathing. Although fast MRI sequences allow artifact free acquisition of individual 2D slices, motion frequently occurs in the acquisition of spatially adjacent slices. Motion correction for each slice is thus critical for the reconstruction of 3D fetal brain MRI. In this paper, we propose a novel multi-task learning framework that adopts a coarse-to-fine strategy to jointly learn the pose estimation parameters for motion correction and tissue segmentation map of each slice in fetal MRI. Particularly, we design a regression-based segmentation loss as a deep supervision to learn anatomically more meaningful features for pose estimation and segmentation. In the coarse stage, a U-Net-like network learns the features shared for both tasks. In the refinement stage, to fully utilize the anatomical information, signed distance maps constructed from the coarse segmentation are introduced to guide the feature learning for both tasks. Finally, iterative incorporation of the signed distance maps further improves the performance of both regression and segmentation progressively. Experimental results of cross-validation across two different fetal datasets acquired with different scanners and imaging protocols demonstrate the effectiveness of the proposed method in reducing the pose estimation error and obtaining superior tissue segmentation results simultaneously, compared with state-of-the-art methods.

In-utero fetal MRI is emerging as an important tool in the diagnosis and analysis of the developing human brain. Automatic segmentation of the developing fetal brain is a vital step in the quantitative analysis of prenatal neurodevelopment both in the research and clinical context. However, manual segmentation of cerebral structures is time-consuming and prone to error and inter-observer variability. Therefore, we organized the Fetal Tissue Annotation (FeTA) Challenge in 2021 in order to encourage the development of automatic segmentation algorithms on an international level. The challenge utilized FeTA Dataset, an open dataset of fetal brain MRI reconstructions segmented into seven different tissues (external cerebrospinal fluid, gray matter, white matter, ventricles, cerebellum, brainstem, deep gray matter). 20 international teams participated in this challenge, submitting a total of 21 algorithms for evaluation. In this paper, we provide a detailed analysis of the results from both a technical and clinical perspective. All participants relied on deep learning methods, mainly U-Nets, with some variability present in the network architecture, optimization, and image pre- and post-processing. The majority of teams used existing medical imaging deep learning frameworks. The main differences between the submissions were the fine tuning done during training, and the specific pre- and post-processing steps performed. The challenge results showed that almost all submissions performed similarly. Four of the top five teams used ensemble learning methods. However, one team's algorithm performed significantly superior to the other submissions, and consisted of an asymmetrical U-Net network architecture. This paper provides a first of its kind benchmark for future automatic multi-tissue segmentation algorithms for the developing human brain in utero.

Contrastive learning, which aims to capture general representation from unlabeled images to initialize the medical analysis models, has been proven effective in alleviating the high demand for expensive annotations. Current methods mainly focus on instance-wise comparisons to learn the global discriminative features, however, pretermitting the local details to distinguish tiny anatomical structures, lesions, and tissues. To address this challenge, in this paper, we propose a general unsupervised representation learning framework, named local discrimination (LD), to learn local discriminative features for medical images by closely embedding semantically similar pixels and identifying regions of similar structures across different images. Specifically, this model is equipped with an embedding module for pixel-wise embedding and a clustering module for generating segmentation. And these two modules are unified by optimizing our novel region discrimination loss function in a mutually beneficial mechanism, which enables our model to reflect structure information as well as measure pixel-wise and region-wise similarity. Furthermore, based on LD, we propose a center-sensitive one-shot landmark localization algorithm and a shape-guided cross-modality segmentation model to foster the generalizability of our model. When transferred to downstream tasks, the learned representation by our method shows a better generalization, outperforming representation from 18 state-of-the-art (SOTA) methods and winning 9 out of all 12 downstream tasks. Especially for the challenging lesion segmentation tasks, the proposed method achieves significantly better performance.

Anomaly detection in fundus images remains challenging due to the fact that fundus images often contain diverse types of lesions with various properties in locations, sizes, shapes, and colors. Current methods achieve anomaly detection mainly through reconstructing or separating the fundus image background from a fundus image under the guidance of a set of normal fundus images. The reconstruction methods, however, ignore the constraint from lesions. The separation methods primarily model the diverse lesions with pixel-based independent and identical distributed (i.i.d.) properties, neglecting the individualized variations of different types of lesions and their structural properties. And hence, these methods may have difficulty to well distinguish lesions from fundus image backgrounds especially with the normal personalized variations (NPV). To address these challenges, we propose a patch-based non-i.i.d. mixture of Gaussian (MoG) to model diverse lesions for adapting to their statistical distribution variations in different fundus images and their patch-like structural properties. Further, we particularly introduce the weighted Schatten p-norm as the metric of low-rank decomposition for enhancing the accuracy of the learned fundus image backgrounds and reducing false-positives caused by NPV. With the individualized modeling of the diverse lesions and the background learning, fundus image backgrounds and NPV are finely learned and subsequently distinguished from diverse lesions, to ultimately improve the anomaly detection. The proposed method is evaluated on two real-world databases and one artificial database, outperforming the state-of-the-art methods.

Existing object re-identification methods usually utilize backbone networks developed based on classification tasks to obtain the final object features. However, these backbone networks lack a unique mechanism to explore discriminative feature representation and handle rich scale changes. For that, a novel hierarchical similarity graph module (HSGM) is proposed to relieve the conflict of backbone networks and mine the discriminative features. Specifically, the proposed HSGM constructs a rich hierarchical graph to explore the pairwise relationships among global-local and local-local. Then, in each hierarchical graph, the HSGM regards local features extracted from different locations as nodes and utilizes the similarity scores between nodes to construct a similarity graph. During the HSGM's propagation, a learnable parameter is reweighted at each spatial position to optimize the correlation between adjacent nodes. Besides, the HSGM can be readily inserted into backbone networks at any depth to improve object discrimination. Extensive experiments on two large-scale object datasets (i.e., VeRi776 and Market-1501) demonstrate that the proposed HSGM is superior to state-of-the-art object re-identification approaches.

Recent works show the possibility of transferring the CLIP (Contrastive Language-Image Pretraining) model for video-text retrieval with promising performance. However, due to the domain gap between static images and videos, CLIP-based video-text retrieval models with interaction-based matching perform far worse than models with representation-based matching. In this paper, we propose a novel image animation strategy to transfer the image-text CLIP model to video-text retrieval effectively. By imitating the video shooting components, we convert widely used image-language corpus to synthesized video-text data for pretraining. To reduce the time complexity of interaction matching, we further propose a coarse to fine framework which consists of dual encoders for fast candidates searching and a cross-modality interaction module for fine-grained re-ranking. The coarse to fine framework with the synthesized video-text pretraining provides significant gains in retrieval accuracy while preserving efficiency. Comprehensive experiments conducted on MSR-VTT, MSVD, and VATEX datasets demonstrate the effectiveness of our approach.

This paper relates the post-analysis of the first edition of the HEad and neCK TumOR (HECKTOR) challenge. This challenge was held as a satellite event of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2020, and was the first of its kind focusing on lesion segmentation in combined FDG-PET and CT image modalities. The challenge's task is the automatic segmentation of the Gross Tumor Volume (GTV) of Head and Neck (H&N) oropharyngeal primary tumors in FDG-PET/CT images. To this end, the participants were given a training set of 201 cases from four different centers and their methods were tested on a held-out set of 53 cases from a fifth center. The methods were ranked according to the Dice Score Coefficient (DSC) averaged across all test cases. An additional inter-observer agreement study was organized to assess the difficulty of the task from a human perspective. 64 teams registered to the challenge, among which 10 provided a paper detailing their approach. The best method obtained an average DSC of 0.7591, showing a large improvement over our proposed baseline method and the inter-observer agreement, associated with DSCs of 0.6610 and 0.61, respectively. The automatic methods proved to successfully leverage the wealth of metabolic and structural properties of combined PET and CT modalities, significantly outperforming human inter-observer agreement level, semi-automatic thresholding based on PET images as well as other single modality-based methods. This promising performance is one step forward towards large-scale radiomics studies in H&N cancer, obviating the need for error-prone and time-consuming manual delineation of GTVs.

Segmentation of hepatic vessels from 3D CT images is necessary for accurate diagnosis and preoperative planning for liver cancer. However, due to the low contrast and high noises of CT images, automatic hepatic vessel segmentation is a challenging task. Hepatic vessels are connected branches containing thick and thin blood vessels, showing an important structural characteristic or a prior: the connectivity of blood vessels. However, this is rarely applied in existing methods. In this paper, we segment hepatic vessels from 3D CT images by utilizing the connectivity prior. To this end, a graph neural network (GNN) used to describe the connectivity prior of hepatic vessels is integrated into a general convolutional neural network (CNN). Specifically, a graph attention network (GAT) is first used to model the graphical connectivity information of hepatic vessels, which can be trained with the vascular connectivity graph constructed directly from the ground truths. Second, the GAT is integrated with a lightweight 3D U-Net by an efficient mechanism called the plug-in mode, in which the GAT is incorporated into the U-Net as a multi-task branch and is only used to supervise the training procedure of the U-Net with the connectivity prior. The GAT will not be used in the inference stage, and thus will not increase the hardware and time costs of the inference stage compared with the U-Net. Therefore, hepatic vessel segmentation can be well improved in an efficient mode. Extensive experiments on two public datasets show that the proposed method is superior to related works in accuracy and connectivity of hepatic vessel segmentation.

Total anomalous pulmonary venous connection (TAPVC) is a rare but mortal congenital heart disease in children and can be repaired by surgical operations. However, some patients may suffer from pulmonary venous obstruction (PVO) after surgery with insufficient blood supply, necessitating special follow-up strategy and treatment. Therefore, it is a clinically important yet challenging problem to predict such patients before surgery. In this paper, we address this issue and propose a computational framework to determine the risk factors for postoperative PVO (PPVO) from computed tomography angiography (CTA) images and build the PPVO risk prediction model. From clinical experiences, such risk factors are likely from the left atrium (LA) and pulmonary vein (PV) of the patient. Thus, 3D models of LA and PV are first reconstructed from low-dose CTA images. Then, a feature pool is built by computing different morphological features from 3D models of LA and PV, and the coupling spatial features of LA and PV. Finally, four risk factors are identified from the feature pool using the machine learning techniques, followed by a risk prediction model. As a result, not only PPVO patients can be effectively predicted but also qualitative risk factors reported in the literature can now be quantified. Finally, the risk prediction model is evaluated on two independent clinical datasets from two hospitals. The model can achieve the AUC values of 0.88 and 0.87 respectively, demonstrating its effectiveness in risk prediction.

Ovarian cancer is a malignant tumor that poses a serious threat to women’s lives. Computer-aided diagnosis (CAD) systems can classify the type of ovarian tumors, but few of them can provide exactly the location information of ovarian cancer cells. Recently, deep learning technology becomes hot for automatic detection of cancer cells, particularly for detecting their locations. In this work, we propose a novel end-to-end network YOLO-OC (Ovarian cancer) model, which can extract the characteristics of ovarian cancer more efficiently. In our method, deformable convolution is used to enhance the model’s ability to learn geometric deformation in space. Squeeze-and-Excitation (SE) module is proposed to automatically learn the importance of different channel features. Data experiments are conducted on datasets collected from The Affiliated Hospital of Qingdao University Medical College, China. Experimental results show that our YOLO-OC model achieves 91.83%, 85.66% and 73.82% on mean average precision [email protected], [email protected] and mAP@[.5,.95], respectively, which performs better than Faster R-CNN, SSD and RetinaNet on both accuracy and efficiency.

Segmentation of colorectal cancerous regions from 3-D magnetic resonance (MR) images is a crucial procedure for radiotherapy. Automatic delineation from 3-D whole volumes is in urgent demand yet very challenging. Drawbacks of existing deep-learning-based methods for this task are two-fold: 1) extensive graphics processing unit (GPU) memory footprint of 3-D tensor limits the trainable volume size, shrinks effective receptive field, and therefore, degrades speed and segmentation performance and 2) in-region segmentation methods supported by region-of-interest (RoI) detection are either blind to global contexts, detail richness compromising, or too expensive for 3-D tasks. To tackle these drawbacks, we propose a novel encoder-decoder-based framework for 3-D whole volume segmentation, referred to as 3-D RoI-aware U-Net (3-D RU-Net). 3-D RU-Net fully utilizes the global contexts covering large effective receptive fields. Specifically, the proposed model consists of a global image encoder for global understanding-based RoI localization, and a local region decoder that operates on pyramid-shaped in-region global features, which is GPU memory efficient and thereby enables training and prediction with large 3-D whole volumes. To facilitate the global-to-local learning procedure and enhance contour detail richness, we designed a dice-based multitask hybrid loss function. The efficiency of the proposed framework enables an extensive model ensemble for further performance gain at acceptable extra computational costs. Over a dataset of 64 T2-weighted MR images, the experimental results of four-fold cross-validation show that our method achieved 75.5% dice similarity coefficient (DSC) in 0.61 s per volume on a GPU, which significantly outperforms competing methods in terms of accuracy and efficiency. The code is publicly available.

Detecting various types of cells in and around the tumor matrix holds a special significance in characterizing the tumor micro-environment for cancer prognostication and research. Automating the tasks of detecting, segmenting, and classifying nuclei can free up the pathologists' time for higher value tasks and reduce errors due to fatigue and subjectivity. To encourage the computer vision research community to develop and test algorithms for these tasks, we prepared a large and diverse dataset of nucleus boundary annotations and class labels. The dataset has over 46,000 nuclei from 37 hospitals, 71 patients, four organs, and four nucleus types. We also organized a challenge around this dataset as a satellite event at the International Symposium on Biomedical Imaging (ISBI) in April 2020. The challenge saw a wide participation from across the world, and the top methods were able to match inter-human concordance for the challenge metric. In this paper, we summarize the dataset and the key findings of the challenge, including the commonalities and differences between the methods developed by various participants. We have released the MoNuSAC2020 dataset to the public.

Automatic rib fracture recognition from chest X-ray images is clinically important yet challenging due to weak saliency of fractures. Weakly Supervised Learning (WSL) models recognize fractures by learning from large-scale image-level labels. In WSL, Class Activation Maps (CAMs) are considered to provide spatial interpretations on classification decisions. However, the high-responding regions, namely Supporting Regions of CAMs may erroneously lock to regions irrelevant to fractures, which thereby raises concerns on the reliability of WSL models for clinical applications. Currently available Mixed Supervised Learning (MSL) models utilize object-level labels to assist fitting WSL-derived CAMs. However, as a prerequisite of MSL, the large quantity of precisely delineated labels is rarely available for rib fracture tasks. To address these problems, this paper proposes a novel MSL framework. Firstly, by embedding the adversarial classification learning into WSL frameworks, the proposed Biased Correlation Decoupling and Instance Separation Enhancing strategies guide CAMs to true fractures indirectly. The CAM guidance is insensitive to shape and size variations of object descriptions, thereby enables robust learning from bounding boxes. Secondly, to further minimize annotation cost in MSL, a CAM-based Active Learning strategy is proposed to recognize and annotate samples whose Supporting Regions cannot be confidently localized. Consequently, the quantity demand of object-level labels can be reduced without compromising the performance. Over a chest X-ray rib-fracture dataset of 10966 images, the experimental results show that our method produces rational Supporting Regions to interpret its classification decisions and outperforms competing methods at an expense of annotating 20% of the positive samples with bounding boxes.

Early diagnosis and continuous monitoring of patients suffering from eye diseases have been major concerns in the computer-aided detection techniques. Detecting one or several specific types of retinal lesions has made a significant breakthrough in computer-aided screen in the past few decades. However, due to the variety of retinal lesions and complex normal anatomical structures, automatic detection of lesions with unknown and diverse types from a retina remains a challenging task. In this paper, a weakly supervised method, requiring only a series of normal and abnormal retinal images without need to specifically annotate their locations and types, is proposed for this task. Specifically, a fundus image is understood as a superposition of background, blood vessels, and background noise (lesions included for abnormal images). Background is formulated as a low-rank structure after a series of simple preprocessing steps, including spatial alignment, color normalization, and blood vessels removal. Background noise is regarded as stochastic variable and modeled through Gaussian for normal images and mixture of Gaussian for abnormal images, respectively. The proposed method encodes both the background knowledge of fundus images and the background noise into one unique model, and corporately optimizes the model using normal and abnormal images, which fully depict the low-rank subspace of the background and distinguish the lesions from the background noise in abnormal fundus images. Experimental results demonstrate that the proposed method is of fine arts accuracy and outperforms the previous related methods.

The 2D transfer function based on scalar value and gradient magnitude (SG-TF) is popularly used in volume rendering. However, it is plagued by the boundary-overlapping problem: different structures with similar attributes have the same region in SG-TF space, and their boundaries are usually connected. The SG-TF thus often fails in separating these structures (or their boundaries) and has limited ability to classify different objects in real-world 3D images. To overcome such a difficulty, we propose a novel method for boundary separation by integrating spatial connectivity computation of the boundaries and set operations on boundary voxels into the SG-TF. Specifically, spatial positions of boundaries and their regions in the SG-TF space are computed, from which boundaries can be well separated and volume rendered in different colors. In the method, the boundaries are divided into three classes and different boundary-separation techniques are applied to them, respectively. The complex task of separating various boundaries in 3D images is then simplified by breaking it into several small separation problems. The method shows good object classification ability in real-world 3D images while avoiding the complexity of high-dimensional transfer functions. Its effectiveness and validation is demonstrated by many experimental results to visualize boundaries of different structures in complex real-world 3D images.

SUMMARY Trust model plays an important role in ensuring the security of interactions in peer‐to‐peer (P2P) systems where a peer's trust evaluation depends on the interaction experience of its own and recommendation information from other peers. However, current trust models have limitations in solving not only the issues of time efficiency of direct interaction information but also the reliability and inconsistency of recommendation information. In this paper, we propose a Dempster‐Shafer evidence theory based trust model (ETTM) for P2P systems. The primary goal of ETTM is to be able to address information uncertainty and conflicting recommendation problems in a reputation‐based P2P environment. To make D‐S theory fits into P2P applications, we creatively revise the combination rules and achieve greatly improved results. To further improve the accuracy and performance, ETTM filters out noisy referrals if they are not compatible with most other evidence. In addition, a feedback‐based probabilistic searching algorithm is proposed to find the referrals with improved searching success rate and lowered network traffic. Experimental results show ETTM has a clear advantage in aggregating recommendation information. Moreover, ETTM is more robust and can generate a higher successful transaction rate than some other existing frameworks. Copyright © 2013 John Wiley & Sons, Ltd.

In this paper we propose a novel and easy to use 3D reconstruction method. With the method, users only need to specify a small boundary surface patch in a 2D section image, and then an entire continuous implicit boundary surface (CIBS) can be automatically reconstructed from a 3D image. In the method, a hierarchical tracing strategy is used to grow the known boundary surface patch gradually in the 3D image. An adaptive detection technique is applied to detect boundary surface patches from different local regions. The technique is based on both context dependence and adaptive contrast detection as in the human vision system. A recognition technique is used to distinguish true boundary surface patches from the false ones in different cubes. By integrating these different approaches, a high-resolution CIBS model can be automatically reconstructed by adaptively expanding the small boundary surface patch in the 3D image. The effectiveness of our method is demonstrated by its applications to a variety of real 3D images, where the CIBS with complex shapes/branches and with varying gray values/gradient magnitudes can be well reconstructed. Our method is easy to use, which provides a valuable tool for 3D image visualization and analysis as needed in many applications.

This paper reveals two important characterizations of global exponential stability (GES) of a generic class of continuous-time recurrent neural networks. First, we show that GES of the neural networks can be fully characterized by global asymptotic stability (GAS) of the networks plus the condition that the maximum abscissa of spectral set of Jacobian matrix of the neural networks at the unique equilibrium point is less than zero. This result provides a very useful and direct way to distinguish GES from GAS for the neural networks. Second, we show that when the neural networks have small state feedback coefficients, the supremum of exponential convergence rates (ECRs) of trajectories of the neural networks is exactly equal to the absolute value of the maximum abscissa of spectral set of Jacobian matrix of the neural networks at the unique equilibrium point. Here, the supremum of ECRs indicates the potentially fastest speed of trajectory convergence. The obtained results are helpful in understanding the essence of GES and clarifying the difference between GES and GAS of the continuous-time recurrent neural networks.

We propose a new method for detecting and approximating the boundary surfaces in three-dimensional (3-D) biomedical images. Using this method, each boundary surface in the original 3-D image is normalized as a zero-value isosurface of a new 3-D image transformed from the original 3-D image. A novel computational framework is proposed to perform such an image transformation. According to this framework, we first detect boundary surfaces from the original 3-D image and compute discrete samplings of the boundary surfaces. Based on these discrete samplings, a new 3-D image is constructed for each boundary surface such that the boundary surface can be well approximated by a zero-value isosurface in the new 3-D image. In this way, the complex problem of reconstructing boundary surfaces in the original 3-D image is converted into a task to extract a zero-value isosurface from the new 3-D image. The proposed technique is not only capable of adequately reconstructing complex boundary surfaces in 3-D biomedical images, but it also overcomes vital limitations encountered by the isosurface-extracting method when the method is used to reconstruct boundary surfaces from 3-D images. The performances and advantages of the proposed computational framework are illustrated by many examples from different 3-D biomedical images.