Uncertainty Quantification (UQ) is critical for detecting hallucinations in black-box Large Vision-Language Models (LVLMs). However, prevailing methods like Discrete Semantic Entropy (DSE) are unreliable, as their scores are primarily dominated by the number of semantic clusters. This renders them incapable of distinguishing between benign semantic ambiguity (varied but coherent responses) and severe belief conflict (contradictory responses). We address this limitation by proposing a novel framework rooted in Dempster-Shafer theory of evidence, built on the premise that not all inconsistency is equal. Our method decomposes uncertainty into two complementary metrics: Belief Divergence, which quantifies ambiguity by measuring the separation between viewpoints, and Belief Conflict, which captures direct logical contradictions. Extensive experiments demonstrate that our framework provides a more reliable measure of uncertainty.

Trusted multi-view classification (TMC) aims to improve prediction reliability by integrating evidence from multiple views. Existing TMC methods extract evidence from single view and use a regularization term to shape the evidence distribution. However, existing methods typically enforce a uniform regularization objective across all views, overlooking critical view-specific biases: intra-view class ambiguity caused by confusable features and inter-view quality disparities reflected in evidence uncertainty. To address these issues, we propose an adaptive regularization strategy that enhances robustness on two levels. At the intra-view level, it quantifies feature ambiguity to apply targeted relaxation to confusable classes, preventing over-penalization of inherent uncertainty. At the inter-view level, it evaluates relative view quality to impose stronger constraints on unreliable views and suppress noise from low-quality ones. Extensive experiments across multiple benchmarks demonstrate the superiority and reliability of the proposed method.

Active retrieval-augmented generation (RAG) triggers external knowledge retrieval during generation based on model-side uncertainty signals to support knowledge-intensive, multi-hop reasoning. However, existing methods often retrieve only after producing a complete answer, failing to surface and fill information gaps in time; moreover, relying on a single internal signal as the trigger cannot adequately capture the multifaceted nature of uncertainty. We therefore propose a conflict-aware active RAG framework. We first decompose complex questions into a sequence of step-level sub-problems. At each step, we quantify local distributional uncertainty via a sliding-window peak token entropy, and estimate cross-sample consensus via the variation ratio computed over multiple Monte Carlo samples. After calibrating both signals onto a probabilistic scale, we quantify their conflict using a symmetric, bounded divergence over Bernoulli parameters, and fuse the three quantities into a single uncertainty score that gates retrieval. Experiments demonstrate the effectiveness of our framework.

The fusion of multi-view medical images through deep neural networks is essential for boosting diagnostic precision in the field of medical image analysis. However, the reliability of these diagnostic results is often compromised by imperfections in image views, manifested as noise, artifacts, and data deficits arising from inconsistent diagnostic frequencies. These issues introduce a significant risk when merging medical views in a clinical setting. To address these problems, we introduce the Reliability-Enhanced Multi-view Network (REMNet), a novel framework designed to tackle two critical challenges: 1) reducing misclassification and uncertainty from imperfect view integration, and 2) improving the reliability and interpretability of multi-view medical image predictions. Specifically, REMNet merges information from multiple views into a coherent evidence framework and incorporates a Dirichlet prior within our predictive model to more accurately estimate confidence in predictions. Coupled with a robust fusion strategy and a precise confidence calibration process, REMNet consolidates the diverse strengths of various medical imaging views, reduces the impact of view imperfections, and enhances the reliability of medical imaging diagnostics. The superiority of REMNet is validated through comprehensive theoretical analysis and empirical experiments on multi-view medical image datasets across different modalities.

In the field of computer-aided diagnosis, particularly for tumor diseases, segmentation is a prerequisite and primary step. Multi-modality images become essential for achieving accurate segmentation, which offer critical insights beyond the limitations of single-modality data. However, different modalities and images may suffer from different types of data imperfection, such as intensity non-uniformity, motion artifact, and low quality due to hardware limitations, which challenge image segmentation algorithms. To address this challenge, we propose a Reliable Evidential Discounting Network (REDNet), which is composed of three main modules: 1) the Intra-modality Consistency Evaluation Module (ICEM) measuring the data cohesion within the same modality; 2) the Cross-modality Difference Aggregation Module (CDAM) identifing data discrepancy across modalities; 3) the Discounting Fusion Module (DFM) processing the multi-modality evidence by applying discounting strategies to fuse the data. This approach maintains segmentation accuracy by effectively integrating multi-modality evidence, while discounting the influence of lower-quality data, ensuring reliable results despite the presence of image imperfections. We evaluated REDNet on two distinct datasets, BRATS2021 and an in-house pancreas dataset from Changhai Hospital. REDNet outperforms other methods, particularly in scenarios with imperfect image sources, and achieves reliable results in multi-modality tumor segmentation.

Multi-view classification based on evidence theory aims to enhance result reliability by effectively quantifying prediction uncertainty at the evidence level, particularly when dealing with low-quality views. However, these methods face limitations in real-world applications due to the sensitivity of estimated uncertainty to view distribution, leading to two main issues: 1) difficulty in making clear judgments about whether to trust predictions based on vague uncertainty scores, and 2) the potential negative impact of integrating information from low-quality views on multi-view classification performance. Both limitations compromise the reliability of multi-view decisions. To address these challenges, we introduce an adaptive rejection mechanism based on estimated uncertainty, which is free of data distribution constraints. By integrating this adaptive rejection mechanism into the fusion of multiple views, our method not only indicates whether predictions should be adopted or rejected at the view level but also enhances classification performance by minimizing the impact of unreliable information. The effectiveness of our method is demonstrated through comprehensive theoretical analysis and empirical experiments on various multi-view datasets, establishing its superiority in enhancing the reliability of multi-view classification.

Medical image datasets often exhibit severe long-tailed distributions, making rare disease recognition challenging. Existing models struggle to balance head- and tail-class performance: single-expert models lack flexibility, while multi-expert methods rely on static aggregation without explicit specialization. We propose an uncertainty-aware expert routing framework that integrates explicit expert specialization with adaptive aggregation guided by epistemic uncertainty. Three experts are dedicated to common, balanced, and rare disease groups, respectively, while an Evidential Deep Learning (EDL) branch estimates uncertainty to guide a lightweight router in dynamically weighting expert outputs. This design allows ambiguous cases to receive greater influence from the rare-disease expert. Experiments on ISIC 2019 and HAM10000 demonstrate state-of-the-art results, notably improving tail-class accuracy and calibration while maintaining strong overall performance, validating the effectiveness of uncertainty-driven expert routing for fair and reliable medical image classification.

Clinical disease classification naturally follows a hierarchical reasoning process, where clinicians first distinguish coarse diagnostic categories (e.g., benign vs. malignant) before identifying specific subtypes to enhance diagnostic accuracy. While multi-view imaging offers complementary perspectives, most existing machine learning approaches perform flat decision fusion, ignoring this intrinsic hierarchy and handling inter-view conflicts and uncertainty primarily at the decision-fusion level. To address this gap, we propose the Hierarchical Multi-view Evidence Aggregation Network (H-MEAN), an end-to-end architecture that explicitly mimics clinical decision-making process. H-MEAN consists of three key modules: (i) Multi-View Fusion, which jointly encodes heterogeneous imaging views to capture complementary features; (ii) Hierarchical Tree Alignment, which structurally propagates diagnostic cues across granularity levels to enforce clinical consistency; and (iii) Hierarchical Evidential Aggregation, which leverages evidential deep learning to quantify uncertainty throughout the diagnostic hierarchy. Extensive experiments on two multi-view medical datasets demonstrate that H-MEAN consistently outperforms competitive methods, achieving higher classification accuracy, particularly in challenging settings with missing or noisy views.

Diffusion-Weighted Imaging (DWI) is essential in prostate cancer diagnosis, yet multi-b-value acquisition increases scan time and susceptibility to motion artifacts. Traditional interpolation methods fail to model complex tissue diffusion, and existing deep learning models often compromise lesion contrast and structural fidelity. We propose a novel framework that integrates latent diffusion models with a mask-guided lesionpreserving (MGLP) mechanism to synthesize low-b-value DWI from high-b-value inputs. The model leverages VQ-VAE-based discrete latent representations for feature alignment and employs lesion masks as spatial priors to preserve diagnostic regions during diffusion. A dynamic feature fusion strategy further enhances anatomical-functional consistency. Experiments on clinical datasets demonstrate superior performance over state-of-the-art methods in PSNR, SSIM, and especially lesion-specific SSIMROI metrics. Our approach effectively reduces the need for multi-b-value scans without sacrificing diagnostic reliability, showing potential as a supporting approach to reduce b-value scanning requirements in prostate MRI.

Multi-modal retinal image registration is critical for clinical diagnosis. Conventional methods often neglect the reliability of keypoint matching, which may lead to registration errors that compromise clinical trustworthiness. To address this challenge, we propose a novel trusted retinal image registration method based on Evidential Deep Learning (EDL) with uncertain keypoint matching. First, the detector networks are employed to measure the confidence and uncertainty of each keypoint within the image pairs. Based on the measure, the top k confident keypoints are selected from corresponding images respectively. Second, the descriptor networks are used to extract the features of the keypoints and compute the similarity matrix for keypoint pairs. Crucially, we utilize Dempster-Shafer theory to fuse the uncertainties of keypoints into the similarity calculation, which facilitates prioritizing high-confident keypoint correspondences while suppressing unreliable matches. Finally, based on the similarity matrix with uncertainty measure, a PROSAC-based registration algorithm is implemented to achieve robust keypoint alignment. Experiments on the FIRE and PRIME-FP20 datasets demonstrate that the proposed method achieves state-of-the-art performance. Ablation studies further validate the effectiveness of uncertainty fusion in image registration, yielding a significant improvement of 13.8% in Mean Landmark Error AUC{@}6.25 compared to the previous best method. Besides enhancing accuracy, through measuring the uncertain keypoint matching, the proposed method can produce trusted retinal image registration results for clinicians.

Cross-modal retrieval aims to match videos and texts by mapping them into a shared feature space. Most existing approaches achieve alignment through contrastive learning based on one-to-one supervised pairs. However, these methods rely too much on supervised signals and do not fully use the unsupervised semantic relationships within each modality. As a result, samples that are semantically similar may be spread in the shared space, which hurts retrieval performance. To solve this problem, we propose a method called Leveraging Intra-Modal Consistency for Cross-Modal Alignment and Retrieval (LICA). Our method introduces a consistency constraint between intra-modal similarities and cross-modal similarity distributions. In this way, samples that are close in meaning stay closer together in the shared space. Experiments on standard text-video retrieval benchmarks show that LICA helps optimize the distribution of the cross-modal feature space and improves retrieval accuracy.

Medical image diagnosis inherently involves uncertainty due to artifacts, occlusions, and ambiguous visual patterns, often leading to high inter-observer variability. While deep neural networks offer strong predictive performance, their outputs tend to be overconfident and poorly calibrated, limiting their clinical reliability. We propose Conformal Focal Loss (CFL), a principled approach that leverages the focal loss and the statistical validity of conformal prediction to better characterize diagnostic uncertainty. By emphasizing hard or ambiguous examples, CFL enables more accurate estimation of both predictive confidence and ambiguity. We evaluate CFL on diagnostic tasks using both clean and noise-augmented datasets, demonstrating its ability to effectively identify uncertain cases while maintaining robust classification performance under label noise.

Trusted multi-view classification (TMVC) addresses variations in data quality by evaluating the reliability of each view based on prediction uncertainty at the evidence level, reducing the impact of low-quality views commonly encountered in real-world scenarios. However, existing TMVC methods often struggle to maintain robustness during testing, particularly when integrating noisy or corrupted views. This limitation arises because the evidence collected by TMVC may be unreliable, frequently providing incorrect information due to complex view distributions and optimization challenges, ultimately leading to classification performance degradation. To enhance the robustness of TMVC methods in real-world conditions, we propose a generalized evidence filtering mechanism that is compatible with various fusion strategies commonly used in TMVC, including Belief Constraint Fusion, Aleatory Cumulative Belief Fusion, and Averaging Belief Fusion. Specifically, we frame the identification of unreliable evidence as a multiple testing problem and introduce p-values to control the risk of false identification. By selectively down-weighting unreliable evidence during testing, our mechanism ensures robust fusion and mitigates performance degradation. Both theoretical guarantees and empirical results demonstrate significant improvements in the classification performance of TMVC methods, supporting their reliable application in challenging, real-world environments.

Feature correspondence, particularly distinguishing inliers (true matches) from outliers (false matches), remains a core challenge in geometric computer vision. We present HAT-Match, a novel Graph Transformer framework with Hybrid Attention for two-view correspondence pruning. HAT-Match integrates three types of attention mechanismsself-attention for modeling pairwise dependencies, SE-based channel attention for emphasizing salient feature channels, and global global structure-aware for capturing structure-aware consistency. To model local geometric relationships, we first generate coarse clusters using a permutation-equivariant graph pooling and unpooling mechanism, which serves as an initial grouping of potentially consistent correspondences. Based on the resulting embeddings, we further construct local graphs using a DGCNN-style k-nearest neighbor (KNN) strategy, enabling the modeling of fine-grained local dependencies. These local graphs are then passed through a hybrid attention module that jointly encodes local and global contextual features. To refine correspondence confidence, we apply graph Laplacian-based attention over the global graph, enhancing discriminative feature propagation. The entire architecture is integrated into a progressive pruning framework that iteratively removes outliers and updates correspondence weights. Extensive experiments demonstrate that HAT-Match achieves state-of-the-art results across various challenging tasks, including relative pose estimation and visual localization, on both indoor and outdoor datasets.

Trustworthy multi-view classification is essential for safety-critical applications, yet existing methods often fail when view quality degrades at test time due to noise or domain shifts. We propose a novel test-time adaptive fusion framework that leverages Conformal Prediction (CP) to transform pre-trained single-view classifier outputs into robust evidence representations. By integrating CP with Dempster-Shafer Theory, our approach dynamically generates well-calibrated evidence for each view, enabling reliable fusion under arbitrary view conditions without retraining. Extensive experiments demonstrate that our method achieves competitive performance on clean data and significantly outperforms prior methods in the presence of view corruption, offering a practical solution for robust multi-view classification in real-world scenarios.

Gradient-based methods provide a principled framework for Active Learning (AL), effectively quantifying a sample’s informativeness through the uncertainty captured from the model’s internal dynamics. However, their effectiveness is often hindered by the failure to disentangle epistemic and aleatoric uncertainty. This can lead to the suboptimal selection of noisy outliers over truly informative samples. To address this, we propose Dual-View Gradient Probes (DVGradProb), a framework that disentangles these uncertainties by probing gradients from two complementary spaces. Specifically, epistemic uncertainty is captured from the parameter space and introduce a novel metric for aleatoric uncertainty from the feature space. These metrics are integrated into a ratio-based acquisition function designed to filter out noisy outliers while prioritizing truly informative samples, those with high epistemic but low aleatoric uncertainty. Experiments on MNIST, CIFAR-10, and SVHN demonstrate that DVGradProb robustly outperforms baselines, validating the effectiveness of this uncertainty disentanglement strategy.

Correspondence learning is a crucial component in multiview geometry and computer vision. The presence of heavy outliers (mismatches) consistently renders the matching problem to be highly challenging. In this article, we revisit the benefits of local consensus (LC) in traditional feature matching and introduce the concept of LC to design a trainable neural network capable of capturing the underlying correspondences. This network is named the LC transformer (LCT) and is specifically tailored for wide-baseline stereo applications. Our network architecture comprises three distinct operations. To establish the neighbor topology, we employ a dynamic graph-based embedding layer as the initial step. Subsequently, these local topologies serve as guidance for the multihead self-attention layer, enabling it to extract a more extensive contextual understanding through channel attention (CA). Following this, order-aware graph pooling is applied to extract the global context information from the embedded LC. Through the experimental analysis, the ablation study reveals that PointNet-like learning models can, indeed, benefit from the incorporation of LC. The proposed model achieves state-of-the-art performance in both challenging scenes, namely, the YFCC100M outdoor and SUN3D indoor environments, even in the presence of more than 90% outliers.

In recent years, multi-view learning has been widely applied in medical imaging. However some low-quality data obtained may contain noise and artifacts, leading to unreliable decisions. Several trustworthy multi-view learning methods have been proposed to assess the uncertainty of decisions. In the diagnosis of pancreatic tumor subtypes (PTS), the subtle differences between subtypes make them difficult to distinguish. Without the guidance of clinical prior knowledge, these method often struggle to capture the key features specific to each subtype. To address these challenges, we propose a novel trustworthy multi-view learning method. By analyzing the potential malignancy risks of PTS, we classify all subtypes into three broad categories. This coarse-grained vague classification informed by clinical prior knowledge is introduced into the neural network as auxiliary information to generate augmented opinions. Additionally, to compensate for the lack of global features, we introduce a global view to supplement the feature space and strengthen multi-view learning. Finally, using an uncertainty-based combination rule, the augmented opinions extracted from multiple views are reliably combined for the final diagnosis. We conduct a series of experiments comparing our approach with the state-of-the-art multi-view learning methods. The experimental results validate the effectiveness of our proposed method, demonstrating its superior performance and trustworthiness.

Accurate segmentation of brain tumors is crucial for effective diagnosis and treatment planning. Magnetic Resonance Imaging (MRI) with multiple modalities provides comprehensive biological information for tumor diagnosis. However, the absence of certain modalities degrades segmentation performance. To address this challenge, we propose an Evidence Harmonization Network (EHNet), which is designed for brain tumor segmentation in scenarios with missing modalities. Firstly, we use subjective logic to model uncertainty, replacing missing modality evidence with averaged evidence from available modality. Secondly, we introduce the Evidence Harmonization Module, which captures the distinction and consensus among modalities and calculates a harmonization factor. It enables the network to dynamically extract effective features in missing modalities scenarios and fully utilizes the strengths of each available modalities for final segmentation. Finally, the Dynamic Fusion Module adjusts the evidence with the harmonization factor and fuses it to produce the final segmentation results. We validate our method on the widely used Brats2018 dataset, demonstrating its superior performance compared to existing methods in missing modalities scenarios. Our approach not only achieves accurate segmentation results but also provides reliable uncertainty estimation.

Uncertainty-aware multi-view deep classification methods have markedly improved the reliability of results amidst the challenges posed by noisy multi-view data, primarily by quantifying the uncertainty of predictions. Despite their efficacy, these methods encounter limitations in real-world applications: 1) They are limited to providing a single class prediction per instance, which can lead to inaccuracies when dealing with samples that are difficult to classify due to inconsistencies across multiple views. 2) While these methods offer a quantification of prediction uncertainty, the magnitude of such uncertainty often varies with different datasets, leading to confusion among decision-makers due to the lack of a standardized measure for uncertainty intensity. To address these issues, we introduce Conformalized Multi-view Deep Classification (CMDC), a novel method that generates set-valued rather than single-valued predictions and integrates uncertain predictions as an explicit class category. Through end-to-end training, CMDC minimizes the size of prediction sets while guaranteeing that the set-valued predictions contain the true label with a user-defined probability, building trust in decision-making. The superiority of CMDC is validated through comprehensive theoretical analysis and empirical experiments on various multi-view datasets.

Evidential Deep Learning (EDL), grounded in Evidence Theory and Subjective Logic (SL), provides a robust framework to estimate uncertainty for out-of-distribution (OOD) detection alongside traditional classification probabilities.However, the EDL framework is constrained by its focus on evidence that supports only single categories, neglecting the other collective evidences that could corroborate multiple in-distribution categories. This limitation leads to a diminished estimation of uncertainty and a subsequent decline in OOD detection performance.Additionally, EDL encounters the vanishing gradient problem within its fully-connected layers, further degrading classification accuracy.To address these issues, we introduce hyper-domain and propose Hyper-opinion Evidential Deep Learning (HEDL). HEDL extends the evidence modeling paradigm by explicitly integrating sharp evidence, which supports a singular category, with vague evidence that accommodates multiple potential categories.Additionally, we propose a novel opinion projection mechanism that translates hyper-opinion into multinomial-opinion, which is then optimized within the EDL framework to ensure precise classification and refined uncertainty estimation.HEDL integrates evidences across various categories to yield a holistic evidentiary foundation for achieving superior OOD detection. Furthermore, our proposed opinion projection method effectively mitigates the vanishing gradient issue, ensuring classification accuracy without additional model complexity. Extensive experiments over many datasets demonstrate our proposed method outperforms existing OOD detection methods.

Accurately segmenting tubular structures, such as blood vessels or nerves, holds significant clinical implications across various medical applications. However, existing methods often exhibit limitations in achieving satisfactory topological performance, particularly in terms of preserving connectivity. To address this challenge, we propose a novel deep-learning approach, termed Deep Closing, inspired by the well-established classic closing operation. Deep Closing first leverages an AutoEncoder trained in the Masked Image Modeling (MIM) paradigm, enhanced with digital topology knowledge, to effectively learn the inherent shape prior of tubular structures and indicate potential disconnected regions. Subsequently, a Simple Components Erosion module is employed to generate topology-focused outcomes, which refines the preceding segmentation results, ensuring all the generated regions are topologically significant. To evaluate the efficacy of Deep Closing, we conduct comprehensive experiments on 4 datasets: DRIVE, CHASE_DB1, DCA1, and CREMI. The results demonstrate that our approach yields considerable improvements in topological performance compared with existing methods. Furthermore, Deep Closing exhibits the ability to generalize and transfer knowledge from external datasets, showcasing its robustness and adaptability. The code for this paper has been available at: https://github.com/5k5000/DeepClosing.

Multi-view deep classification expects to obtain better classification performance than using a single view. However, due to the uncertainty and inconsistency of data sources, adding data views does not necessarily lead to the performance improvements in multi-view classification. How to avoid worsening classification performance when adding views is crucial for multi-view deep learning but rarely studied. To tackle this limitation, in this paper, we reformulate the multi-view classification problem from the perspective of safe learning and thereby propose a Safe Multi-view Deep Classification (SMDC) method, which can guarantee that the classification performance does not deteriorate when fusing multiple views. In the SMDC method, we dynamically integrate multiple views and estimate the inherent uncertainties among multiple views with different root causes based on evidence theory. Through minimizing the uncertainties, SMDC promotes the evidences from data views for correct classification, and in the meantime excludes the incorrect evidences to produce the safe multi-view classification results. Furthermore, we theoretically prove that in the safe multi-view classification, adding data views will certainly not increase the empirical risk of classification. The experiments on various kinds of multi-view datasets validate that the proposed SMDC method can achieve precise and safe classification results.

Real-world classification tasks often show an extremely imbalanced problem. The extreme imbalance will cause a strong bias that the decision boundary of the classifier is completely dominated by the categories with abundant samples, which are also called the head categories. Current methods have alleviated the imbalanced impact from mainly three aspects: class re-balance, decoupling and domain adaptation. However, the existing criterion with the winner-take-all strategy still leads to the crowding problem in the eigenspace. The head categories with many samples can extract features more accurately, but occupy most of the eigenspace. The tail categories sharing the rest of the narrow eigenspace are too crowded together to accurately extract features. Above these issues, we propose a novel T-distributed spherical metric for equalized eigenspace in the imbalanced classification, which has the following innovations: 1) We design the T-distributed spherical metric, which has the characteristics of high kurtosis. Instead of the winner-take-all strategy, the T-distributed spherical metric produces a high logit only when the extracted feature is close enough to the category center, without a strong bias against other categories. 2) The T-distributed spherical metric is integrated into the classifier, which is able to equalize the eigenspace for alleviating the crowding issue in the imbalanced problem. The equalized eigenspace by the T-distributed spherical classifier is capable of improving the accuracy of the tail categories while maintaining the accuracy of the head, which significantly promotes the intraclass compactness and interclass separability of features. Extensive experiments on large-scale imbalanced datasets verify our method, which shows superior results in the long-tailed CIFAR-100/-10 with the imbalanced ratio IR = 100/50. Our method also achieves excellent results on the large-scale ImageNet-LT dataset and the iNaturalist dataset with various backbones. In addition, we provide a case study of the real clinical classification of pancreatic tumor subtypes with 6 categories. Among them, the largest number of PDAC accounts for 315 cases, and the least CP has only 8 cases. After 4-fold cross-validation, we achieved a top-1 accuracy of 69.04%.

Correspondence learning is a vital component in multi-view geometry and computer vision. The heavy outliers make the matching problem very challenging. By revisiting the local consensus benefits of traditional feature matching, we introduce the local consensus to design a learnerable neural network to capture underlying correspondences, dubbed the Local Consensus Transformer, for wide-baseline stereo. Specifically, our network architecture consists of three operations. In order to construct the neighbor topology, a dynamic graph-based embedding layer is used first. These local topologies then guide the multihead self-attention layer to mine a greater amount of context through channel attention. After that, order-aware graph pooling is applied to extract global context from the embedded local consensus. Experimentally, the abalation study shows that pointnet-like learning models can benefit from local consensus. The proposed model achieves state-of-the-art performance on both the YFCC100M outdoor and SUN3D indoor challenging scenes with more than 90 percent outliers.

Multi-view deep learning is performed based on the deep fusion of data from multiple sources, i.e. data with multiple views. However, due to the property differences and inconsistency of data sources, the deep learning results based on the fusion of multi-view data may be uncertain and unreliable. It is required to reduce the uncertainty in data fusion and implement the trusted multi-view deep learning. Aiming at the problem, we revisit the multi-view learning from the perspective of opinion aggregation and thereby devise a trusted multi-view deep learning method. Within this method, we adopt evidence theory to formulate the uncertainty of opinions as learning results from different data sources and measure the uncertainty of opinion aggregation as multi-view learning results through evidence accumulation. We prove that accumulating the evidences from multiple data views will decrease the uncertainty in multi-view deep learning and facilitate to achieve the trusted learning results. Experiments on various kinds of multi-view datasets verify the reliability and robustness of the proposed multi-view deep learning method.

Pre-training deep learning models on ImageNet and transferring the models to medical image applications facilitate to improve the medical image analysis and reduce the need for labeled medical image data. However, some images from ImageNet may be fundamentally different from medical images in feature representation and lead to the negative transfer effects. To deal with this issue, we propose a novel strategy based on evidence theory to select reliable instances from ImageNet for medical image domain adaptation. Specifically, we formulate an evidential mass function to measure the ignorance and reliability of the images from ImageNet with respect to the classification tasks of medical images. Through selecting reliable instances with low ignorance degree from ImageNet, we can enhance the transfer performances of deep neural networks in medical image domain adaptation. Moreover, the proposed data selection strategy is independent of specific learning algorithm and can be viewed as a common preprocessing technique. Numerical experiments on tomography images, X-Ray images, and ultrasound images are given to comprehensively demonstrate the effectiveness of the selection strategy.

Labeling cerebral vessels requires domain knowledge in neurology and could be extremely laborious, and there is a scarcity of public annotated cerebrovascular datasets. Traditional machine learning or statistical models could yield decent results on thick vessels with high contrast while having poor performance on those regions of low contrast. In our work, we employ a statistic model as noisy labels and propose a Transformer-based architecture which utilizes Hessian shape prior as soft supervision. It enhances the learning ability of the network to tubular structures, so that the model can make more accurate predictions on refined cerebrovascular segmentation. Furthermore, to combat the overfitting towards noisy labels as model training, we introduce an effective label extension strategy that only calls for a few manual strokes on one sample. These supplementary labels are not used for supervision but only as an indicator to tell where the model keeps the most generalization capability, so as to further guide the model selection in validation. Our experiments are carried out on a public TOF-MRA dataset from MIDAS data platform, and the results demonstrate that our method shows superior performance on cerebrovascular segmentation which achieves Dice of 0.831±0.040 in the dataset.

Abstract Precise segmentation of the lung airways is essential for a quantitative assessment of airway diseases. However, because of the complexity of the airway structure and the different thicknesses of the trachea at different positions, it is extremely difficult to segment the fine bronchial structure using chest computed tomography (CT). Traditional lung airway segmentation methods are generally based on the grayscale, geometric shape of the image, or the use of prior knowledge of anatomy. In recent years, deep learning techniques such as fully convolutional neural networks (FCNs) have achieved a great success in the field of image segmentation. Specifically, the symmetric encoder–decoder network represented by U‐Net has achieved high accuracy in many medical image segmentation tasks. In the airway segmentation challenge task of the 4th International Symposium on Image Computing and Digital Medicine (ISICDM 2020), 9 of the 12 teams participating in the final round used the U‐Net network or one of its other forms, obtaining good results for lung airway segmentation. Methods used to improve the segmentation accuracy include attention mechanisms and multiscale feature information fusion. This article provides a detailed description of the methods used by these 12 teams and analyses their results.

The cross-modality generation is usually a challenging task, especially for the medical image with a huge distribution difference. Micro Computed Tomography (Micro-CT) has high resolution and sharp contrast, which is regarded as the gold standard of tooth structure in dentistry. But Micro-CT has to scan a single isolated tooth in vitro, it cannot provide any information before surgery. Cone Beam Computed Tomography (CBCT) is commonly used in clinical treatment, but its low resolution and blurred border often lead to the deficiency of accurate tooth structure for surgery, such as the pulp and crown. The modality conversion from CBCT to Micro-CT is very difficult due to the huge difference between two modality distributions. Most GAN-based models cannot cope with such distribution differences between modalities. Therefore, we propose a novel cross-modality method for generating Micro-CT from CBCT. First, we propose a variation-based distance to reflect the difference between the two modality distributions. Second, integrating the variation-based distance, we design an end-to-end neural network named Variational Synthesis network (VSnet) to generate the Micro-CT from CBCT. Finally, we compare our method with plentiful state-of-the-art image generation methods. The results show that our proposed method is superior to other existing methods on the Micro-CT generation task, with image quality index SSIM of 0.912 $(\pm 0.061)$, PSNR of 22.451 $(\pm 4.810)$, and NRMSE of 0.261 $(\pm 0.142)$. In terms of the generation of the tooth structure, we also achieve superior results in clinic application. For the Micro-CT generated by VSnet, it is easy to achieve precise segmentations of tooth pulp and crown through simple thresholding, which facilitates the diagnosis in dental clinic.

Bladder cancer staging is crucial for operation planning and cancer assessment. Deep Convolutional Neural Networks (DCNNs) have been widely used to classify the bladder tumor images to identify cancer stages. However, the pure image-based deep learning methods over depend on the labeled data training and neglect the clinical priors. Human doctors judge the stage of a bladder tumor through checking whether the tumor infiltrating into bladder wall. The clinical priors of tumor infiltration are helpful to improve the DCNN-based bladder cancer staging and make the predictions coincide with the law of medicine. To involve clinical priors into deep learning for cancer staging, we propose a DCNN model with prior evidence to classify medical images of bladder tumors. Specifically, we measure the degree of tumor infiltrating into bladder wall to construct the prior evidence and integrate the prior evidence into the image-based prediction with evidential deep neural networks. We analyze the learning objective and prove that the prior evidences consistent with the ground truth will certainly reduce the prediction error and variance produced by image-based neural networks. The experiments on bladder cancer MR images datasets validate that involving prior evidences is effective to improve the DCNN-based cancer staging.

Unsupervised Image-to-Image generation has obtained many studies recently, where generative adversarial networks (GANs) have developed as an effective model. Cycle consistency or dual learning guides the GAN from aligned image pairs to the unpaired training set and can be applied for many applications based on the image-to-image generation. However, for many tasks, error accumulation, which can be produced in the progress of image reconstruction, affects the realism and quality of the generated images. To eliminate error accumulation and guide the adversarial learning, in this paper, we propose dual-cycle consistent GAN (DucGAN), a novel approach for cross-domain image-to-image generation. The key idea is to introduce dual-cycle learning, which restrains error accumulation. In our method, inter-cycle and extra-cycle are based on the cycle consistency, while the output from intercycle is cast as a new augmented input in extra-cycle. On the one hand, reconstruction loss in extra-cycle can constrain and guide the training. On the other hand, dual-cycle learning is self-augmented. Our extensive experiments on image-to-image generation tasks show that DucGAN is effective and superior to several state-of-the-art GANs.

Segmenting bladder tumors from MR images is important for the early detection and auxiliary diagnosis of bladder cancer. In recent years, Deep Convolutional Neural Networks (DCNNs) have been widely used for bladder tumor segmentation. However, the DCNN-based tumor segmentation methods generally over-depend on the model training with labeled data, and neglect the priors that can be learned from data or domain experts. The priors of tumor size and location are very helpful to improve the segmentation precision of bladder tumors, especially for the situation when the labeled images are insufficient to cover the diversity. To tackle the problem, we proposed a method to integrate priors into DCNN-based bladder tumor segmentation in this paper. The priors consist of both specific prior from data and general prior from domain knowledge. Specifically, we add a decoding path into the segmentation network to generate the specific prior, and formulate the general prior of domain knowledge to guide the generation of specific prior. Comprehensive experiments on massive bladder MR images clearly validate the effectiveness of the proposed bladder tumor segmentation method with priors.

Chinese traditional music has been proved to be effective in emotion regulation for thousands of years. Five different groups of Chinese traditional music which have been proved can regulate different emotions (Angry, Depressed, Feverish, Desperate, Sorrowful) in the literature. 54 audios features are extracted by using the Librosa library for each music group. Five features are manually selected using histogram analysis which show significant difference between the five groups of music. Combined with KNN, SVM and Deep forest classification algorithms, the five manually selected audio features are shown to have better classification performance than traditional feature selection algorithms, like PCA and LDA. We hypothesize that these five significant audio features may be the underlying basis why so such music can effectively perform emotion regulation. Based on this classification models, prototype emotion regulation music recommendation interface (TJ-ERMR) was built that can be used for music therapy. In the future, we will use this classification model to find more music to expand the initial repertoire of our music recommendation system.

While point set registration has been studied in many areas of computer vision for decades, registering points encountering different degradations remains a challenging problem. In this article, we introduce a robust point pattern matching method, termed spatially coherent matching (SCM). The SCM algorithm consists of recovering correspondences and learning nonrigid transformations between the given model and scene point sets while preserving the local neighborhood structure. Precisely, the proposed SCM starts with the initial matches that are contaminated by degradations (e.g., deformation, noise, occlusion, rotation, multiview, and outliers), and the main task is to recover the underlying correspondences and learn the nonrigid transformation alternately. Based on unsupervised manifold learning, the challenging problem of point set registration can be formulated by the Gaussian fields criterion under a local preserving constraint, where the neighborhood structure could be preserved in each transforming. Moreover, the nonrigid transformation is modeled in a reproducing kernel Hilbert space, and we use a kernel approximation strategy to boost efficiency. Experimental results demonstrate that the proposed approach robustly rejecting mismatches and registers complex point set pairs containing large degradations.

Bladder cancer is a malignant disease with substantial morbidity and mortality. Bladder cancer staging is crucial to determine the effective treatments of bladder tumors in clinic. As to the superiority of feature learning, Deep Convolutional Neural Networks (DCNN) are widely used to predict the cancer stage based on medical images. However, most existing DCNN-based cancer staging methods are data-driven and neglect the domain knowledge and experiences of clinicians. Besides, the deep neural networks are short of model interpretability and may lead to risky diagnosis. To tackle the problems, we construct the diagnosis rules of bladder cancer staging based on the clinical experiences of tumor penetration into bladder wall. The diagnosis rules are extracted from Magnetic Resonance (MR) images and further integrated into DCNN for joint identification of tumor stage. The experiments validate that the integrated rules improve the model interpretability and guide DCNN to focus on the regions of tumor penetration and thereby produce precise prediction of cancer staging.

In order to solve the registration problem, we propose a robust method called Spatially Coherent Matching (SCM), where it can get the underlying correspondences from the given putative sets of feature points for robust matching, and estimate the transformation for robust registration. Recovering correct matches and fitting transformations between image pairs are key components in the field of pattern recognition. The proposed SCM starts with a putative correspondence set which is contaminated by degradations (e.g., occlusion, deformation, rotation, and outliers), and the main goal is to identify the true correspondences and estimate the underlying transformation. Then we formulate this challenging problem by the spatially coherent matching model with a robust exponential distance loss and a spatial constraint. Based on the regularization theory, SCM preserves the topological structure of the adjacent features. Moreover, a sparse approximation strategy is used to improve the efficiency. Finally, the experimental results reveal that the proposed method outperforms current state-of-the-art methods in most test scenarios on several real image datasets and synthesized datasets.

The ensemble of feature selections facilitates to improve data generalization for learning tasks. However, existing feature selection ensemble methods have the following drawbacks. First, focusing on the numerical data, the works on the feature selection ensemble for symbolic and mixed-type data are very limited. Second, the voting-based ensemble strategies tend to select the top significant features but the consistency between the ensemble result and the diverse feature selections cannot be guaranteed. Aiming to handle these problems, we propose an feature selection ensemble method based on Analytic Hierarchy Process (AHP) in this paper. The AHP-based ensemble method can integrate diverse feature selections into a consistent one under the multiple criteria of feature discernibility and independence. Moreover, the ensemble methodology is helpful to implement the feature selection on distributed data and involve domain knowledge through extending criteria. Experimental results validate that the proposed ensemble method of feature selections is effective for symbolic data classification.

Multimodal retinal image registration plays an important role in medical image analysis. In this field, retinal images from different modalities are aligned together to achieve a more evaluable fusion image for diagnoses. One of the challenging problem solved in this paper is the low success rate in multimodal retinal image registration. An automated point set registration framework is proposed to solve the problem. The framework includes three parts: feature point extraction and robust initial point matching, matching postprocessing, adaptive mismatches removing and transformation estimation. The experimental results show that our proposed framework is robust to outliers and repeated pattern and it obtains a more stable and accurate result than state-of-the-art methods.

Most image segmentation methods based on active contour model are sensitive to the contour initialization. For the images of complex contents, it is difficult to initialize the contour properly and the biased initialization may lead to low-quality segmentation results. Aiming to tackle this problem, we propose an ensemble strategy to improve the contour-based segmentation. The optimal segmentation ensemble is obtained through maximizing the weighted mutual information between the probability distributions of multiple segmentation results. Experimental results validate that the ensemble of contour-based segmentation is robust to the biased initialization and produces stable and precise results for the images of complex contents.

Estimating transformations from degraded point sets is necessary for many computer vision and pattern recognition applications. In this paper, we propose a robust non-rigid point set registration method based on spatially constrained context-aware Gaussian fields. We first construct a context-aware representation (e.g., shape context) for assignment initialization. Then, we use a graph Laplacian regularized Gaussian fields to estimate the underlying transformation from the likely correspondences. On the one hand, the intrinsic manifold is considered and used to preserve the geometrical structure, and a priori knowledge of the point set is extracted. On the other hand, by using the deterministic annealing, the presented method is extended to a projected high-dimensional feature space, i.e., reproducing kernel Hilbert space through a kernel trick to solve the transformation, in which the local structure is propagated by the coarse-to-fine scaling strategy. In this way, the proposed method gradually recovers much more correct correspondences, and then estimates the transformation parameters accurately and robustly when facing degradations. Experimental results on 2D and 3D synthetic and real data (point sets) demonstrate that the proposed method reaches better performance than the state-of-the-art algorithms.

Point set registration (PSR) is a fundamental problem in computer vision and pattern recognition, and it has been successfully applied to many applications. Although widely used, existing PSR methods cannot align point sets robustly under degradations, such as deformation, noise, occlusion, outlier, rotation, and multi-view changes. This paper proposes context-aware Gaussian fields (CA-LapGF) for nonrigid PSR subject to global rigid and local non-rigid geometric constraints, where a laplacian regularized term is added to preserve the intrinsic geometry of the transformed set. CA-LapGF uses a robust objective function and the quasi-Newton algorithm to estimate the likely correspondences, and the non-rigid transformation parameters between two point sets iteratively. The CA-LapGF can estimate non-rigid transformations, which are mapped to reproducing kernel Hilbert spaces, accurately and robustly in the presence of degradations. Experimental results on synthetic and real images reveal that how CA-LapGF outperforms state-of-the-art algorithms for non-rigid PSR.

Point set registration problem is challenging to solve in the presence of outliers. In this paper, we proposed a registration method based on fuzzy correspondences and kernel density estimation. The main idea of our method is that the moving point set consists of inliers represented using a mixture of Gaussian, and outliers represented via an additional uniform distribution, then we use the fuzzy correspondences to estimate the Gaussian elements in the mixture model. There are four parts of the paper: we formulate the contaminated point set registration problem as a mixture model according to the well known Gaussian mixture model (GMM) based method firstly. Secondly, Gaussian elements are estimated by fuzzy correspondences to increase the registration accuracy efficiently. Thirdly, the optimal transformation between two contaminated point sets is expressed by representation theorem, and solved by EM algorithm iteratively. Finally, we compare our proposed method with several state-of-the-art methods, and the results show that our method gets better performances than the other methods in most tested scenarios.