National audience

Knowledge graphs (KGs) play a central role in enabling structured reasoning for AI applications. Recent research has advanced two prominent extensions of KGs: multimodal KGs, which integrate diverse data sources such as text, images, audio, and video; and temporal KGs, which capture the dynamic evolution of knowledge over time. However, these two paradigms remain largely disjoint—multimodal KGs typically ignore temporal dynamics, while temporal KGs overlook rich multimodal context. To bridge this gap, we introduceFocusKG, the first discrete-time multimodal knowledge graph that unifies four modalities (text, image, audio, and video) with temporal annotations. Built from the Focus news program, FocusKG captures real-world events as temporally grounded multimodal facts, offering new expressiveness for time-sensitive tasks. We further propose aDiscrete-TimeMultimodal Knowledge GraphEmbedding (DTME) method, a novel framework tailored for learning representations over temporal multimodal KGs. DTME is composed of three key modules: (1) Multimodal Preprocessing, which encodes raw inputs from each modality into vector representations using modality-specific encoders; (2) Cross-Modal Graph Enhancer, which captures semantic interplay between entity and relation modalities through role-aware subgraph construction and a modality interaction network; and (3) Multi-Branch Time Aggregation, which injects temporal signals by modeling interactions between time and modality-aware features. Extensive experiments demonstrate FocusKG's value and DTME's efficacy. On link prediction, DTME outperforms state-of-the-art multimodal KG models and temporal KG models. We release FocusKG as a benchmark to foster research in multimodal temporal reasoning.

In the field of medical image processing, vascular image segmentation plays a crucial role in clinical diagnosis, treatment planning, prognosis, and medical decision-making. Accurate and automated segmentation of vascular images can assist clinicians in understanding the vascular network structure, leading to more informed medical decisions. However, manual annotation of vascular images is time-consuming and challenging due to the fine and low-contrast vascular branches, especially in the medical imaging domain where annotation requires specialized knowledge and clinical expertise. Data-driven deep learning models struggle to achieve good performance when only a small number of annotated vascular images are available. To address this issue, this paper proposes a novel Conditional Virtual Imaging (CVI) framework for few-shot vascular image segmentation learning. The framework combines limited annotated data with extensive unlabeled data to generate high-quality images, effectively improving the accuracy and robustness of segmentation learning. Our approach primarily includes two innovations: First, aligned image-mask pair generation, which leverages the powerful image generation capabilities of large pre-trained models to produce high-quality vascular images with complex structures using only a few training images; Second, the Dual-Consistency Learning (DCL) strategy, which simultaneously trains the generator and segmentation model, allowing them to learn from each other and maximize the utilization of limited data. Experimental results demonstrate that our CVI framework can generate high-quality medical images and effectively enhance the performance of segmentation models in few-shot scenarios. Our code will be made publicly available online.

Computed tomography (CT) scanners are widely used to obtain detailed internal images in clinical diagnosis. Highly attenuated metallic implants resulting from strong and energy-dependent attenuation cause metal artifacts in CT scanning. However, current supervised deep network-based metal artifact reduction (MAR) methods hardly generalize in clinical diagnosis and treatment because of difficult acquisition for the paired artifact-affected and artifact-free data. In addition, these deep model-based methods cannot ensure the sinogram-domain data consistency for the exact metal trace inpainting. To address the above problems, we propose an UnsuPervised sinoGRam-domAin Data-consistEnt network for MAR, i.e., UPGRADE-Net. First, UPGRADE-Net fully leverages the prior knowledge to guide the generative conditional diffusion model for fine-grained metal trace inpainting. Second, without the artifact-free ground truth, a deep unsupervised MAR framework in the reverse process is constructed to contextually learn the known background data distribution for the unknown metal trace restoration in sinogram-domain. Third, to further maintain the sinogram-domain data consistency, two physics-based consistency constraint loss functions, including conjugate-ray and accumulation-ray consistency loss, are designed for the conjugate point constraint and the accumulation constraint. The proposed UPGRADE-Net is trained and evaluated on a publicly available dataset and a clinical dataset. Extensive experimental results validate that the proposed method outperforms the state-of-the-art competing methods for MAR.

Single-channel speech separation remains one of the most challenging tasks in the field of speech signal processing. In many situations, such as during epidemics that involve respiratory diseases (e.g., COVID-19 or influenza A), individuals are required to wear masks while communicating. Is it possible to address the challenge of speech separation when the target speaker is wearing a mask? Can audio–visual approaches achieve better speech separation performance than that of audio-only approaches in scenarios where speakers are wearing masks? To address the aforementioned questions, we first construct a large-scale multimodal dataset, termed Speech Separation while Wearing a Mask (SSWM), which includes both the audio modality and the visual modality with masked faces. We explore two strategies for addressing the problem of facial occlusion. One strategy involves utilizing occluded faces—which lack critical visual cues such as mouth movements—directly as supervisory information for self-supervised speech separation; the other strategy involves the use of Wav2Lip to first generate visual information, which is then used as supervisory guidance for self-supervised speech separation. Building upon these two strategies, we propose the SSWM network (SSWMNet), which can flexibly choose to either utilize occluded facial images directly or employ Wav2Lip to generate visual information. The experimental results demonstrate that the proposed speech separation method in which Wav2Lip is used for visual information generation outperforms the approach of utilizing occluded faces directly for self-supervised speech separation. Both proposed audio–visual methods outperform the audio-only speech separation approach, which operates without the aid of visual information. Availability—SSWMNet is available at https://github.com/fanmanqian/SSWMNetwork .

Graph distillation endeavors to transfer knowledge from large, complex teacher models, such as Graph Neural Networks (GNNs), to smaller, more efficient student models such as Multi-Layer Perceptrons (MLPs). Our work is motivated by a critical observation: as the complexity and depth of the teacher GNN increase, the performance of the distilled student MLP models tends to decline significantly. This issue is not limited to a specific distillation method but is a prevalent challenge across various GNN to MLP approaches. To address this gap, we introduce a novel prompt-driven graph distillation framework that enhances the student model’s input space by appending or integrating learned prompts with the original features. These prompts, derived from the teacher’s input, hidden and output knowledge, provide additional context that assists the MLP student in distilling information, thereby circumventing the need to directly capture any teacher’s knowledge into the simplest single-layer MLP. Our empirical experiment on benchmark graph datasets reveals a counter-intuitive phenomenon: the more capable the teacher, the greater the student’s ability to utilize prompts to outperform the teacher. This finding highlights the strength of our approach: a well-prompted student can indeed surpass its teacher, at achieving the best performance with an accuracy of 87.2% on the Cora standard split with a single-layer MLP, while also maintaining efficiency and robustness. In addition to testing on benchmark graph datasets, we applied our framework to the TUH EEG Epilepsy Corpus (TUEP), condensing complex GNN models to simple MLPs while achieving good performance and efficiency in epilepsy classification.

Video question answering (VideoQA) is the challenging task of accurately responding to natural language questions based on a given video. Most previous methods focus on designing complex cross-modal interactions to perform question-oriented video scene mining and semantic reasoning, and utilize straightforward classification and matching strategies with different decoders to forcibly associate the predicted representation with ground-truth answer. However, the limitations of question-oriented reasoning and the overlapping semantic co-occurrences between questions and candidates may cause them to fall into spurious correlation reasoning. In this paper, we propose a Collaborative aware Bidirectional Semantic Reasoning (CBSR) model to alleviate this challenging problem. Specifically, we first propose a collaborative aware adaptive correlation reasoning module to collaboratively mine multi-granularity text-aware critical video scenes and reason about the complex intrinsic correlations between them via bottom-up cross-granularity adaptive aggregation. By progressively performing video reasoning from object-level to frame-level, we can obtain a set of semantically rich critical video representations. Then, we collaboratively decode it together with question and knowledge semantics into an implicit representation through the proposed unified answer semantic collaborated decoding module. Finally, a novel bidirectional semantic reasoning learning strategy is proposed to bridge and strengthen the unique positive semantic correlation between the learned implicit representation and the ground-truth answer, and explicitly alleviate the challenge of overlapping semantic co-occurrence. Benefiting from the same model structure and learning strategy, our method can achieve seamless transfer between Open-Ended and Multi-Choice tasks. Extensive experimental results on seven commonly tested datasets (i.e. MSVD-QA, MSRVTT-QA, NExT-QA, Causal-VidQA, NExT-OOD, ActivityNet-QA and EgoSchema) verify the superior performance of our method and the effectiveness of each reasoning module. We provide our source codes and experimental datasets athttps://github.com/XizeWu/CBSR.

Recent advances in Multimodal Entity Linking leverage multimodal information to link target mentions to corresponding entities. However, existing methods uniformly adopt a “one-size-fits-all” approach, which overlooks the unique requirements of individual samples and fails to adequately balance modality-assisted disambiguation and modality-induced noise. Also, the commonly used separate large-scale visual and text pretrained models for feature extraction do not address inter-modal heterogeneity and the high computational cost of fine-tuning. To resolve these two issues, we introduce a novel approach named Multimodal Entity Linking with Dynamic Modality Selection and Interactive Prompt Learning (DSMIP). First, we design three expert networks that utilize different subsets of modalities tailored to the task and train them individually. Specifically, for the multimodal expert network, we enhance entity and mention feature extraction by updating multimodal prompts and setting up a coupling function to realize the interaction of prompts between modalities. Subsequently, to select the best-suited expert network for each specific sample, we devise a Modality Selection Gating Network to gain the optimal one-hot selection vector by applying a specialized reparameterization technique and a two-stage training process. Experimental results on three public benchmark datasets demonstrate that the proposed DSMIP outperforms all state-of-the-art baselines. The code is released on https://github.com/mayy-seu/DSMIP-code.

In image processing, wavelet transform (WT) offers multiscale image decomposition, generating a blend of low-resolution approximation images and high-resolution detail components. Drawing parallels to this concept, we view feature maps in convolutional neural networks (CNNs) as a similar mix, but uniquely within the channel domain. Inspired by multitask learning (MTL) principles, we propose a wavelet-based dual-task (WDT) framework. This novel framework employs WT in the channel domain to split a single task into two parallel tasks, thereby reforming traditional single-task CNNs into dynamic dual-task networks. Our WDT framework integrates seamlessly with various popular network architectures, enhancing their versatility and efficiency. It offers a more rational approach to resource allocation in CNNs, balancing between low-frequency and high-frequency information. Rigorous experiments on Cifar10, ImageNet, HMDB51, and UCF101 validate our approach's effectiveness. Results reveal significant improvements in the performance of traditional CNNs on classification tasks, and notably, these enhancements are achieved with fewer parameters and computations. In summary, our work presents a pioneering step toward redefining the performance and efficiency of CNN-based tasks through WT.

Deep learning and Convolutional Neural Networks (CNNs) have driven major transformations in diverse research areas. However, their limitations in handling low-frequency in-formation present obstacles in certain tasks like interpreting global structures or managing smooth transition images. Despite the promising performance of transformer struc-tures in numerous tasks, their intricate optimization com-plexities highlight the persistent need for refined CNN en-hancements using limited resources. Responding to these complexities, we introduce a novel framework, the Mul-tiscale Low-Frequency Memory (MLFM) Network, with the goal to harness the full potential of CNNs while keep-ing their complexity unchanged. The MLFM efficiently preserves low-frequency information, enhancing perfor-mance in targeted computer vision tasks. Central to our MLFM is the Low-Frequency Memory Unit (LFMU), which stores various low-frequency data and forms a parallel channel to the core network. A key advantage of MLFM is its seamless compatibility with various prevalent networks, requiring no alterations to their original core structure. Testing on ImageNet demonstrated substantial accuracy improvements in multiple 2D CNNs, including ResNet, MobileNet, EfficientNet, and ConvNeXt. Furthermore, we showcase MLFM's versatility beyond traditional image classification by successfully integrating it into image-to-image translation tasks, specifically in semantic segmenta-tion networks like FCN and U-Net. In conclusion, our work signifies a pivotal stride in the journey of optimizing the ef-ficacy and efficiency of CNNs with limited resources. This research builds upon the existing CNN foundations and paves the way for future advancements in computer vision. Our codes are available at https://github.com/AlphaWuSeu/MLFM.

Multivariate autoregressive modeling is widely considered in neuroscience, especially when effective connectivity is concerned. In high-dimensional space, the conventional least-squares estimation of the autoregressive coefficients is no more consistent, hence the interest in regularizing the solution. Therefore, regularized approaches have been developed such as the Least Absolute Shrinkage Selection Operator (LASSO) promoting sparsity, which performs well provided that it is combined with the extended Bayesian Information Criterion (eBIC) to jointly estimate the MVAR model order and penalty parameter. Unfortunately, this need for eBIC requires much more computation time, making such approaches unsuitable for identifying high-dimensional MVAR models. The method proposed in this letter, named SOCAR (Simultaneous identification of the Order and the Coefficients of multivariate AutoRegressive models), estimates both the order and the coefficients of the model by combining mixed-norm regularization with classical sparse priors (e.g. LASSO). Weighted penalties are minimized in order to drive more easily sparsity to the right place. Experiments carried out on simulated signals show that SOCAR offers a good compromise between performance and numerical complexity.

By seamlessly integrating wavelet transforms into the image patching stage of ViT, we leverage the power of multi-level wavelet transforms to decompose images into a diverse array of frequency-domain features. These features, integrated with spatial characteristics at equivalent scales, enrich image details, enhancing ViT's proficiency in delineating intricate textures and distinct edges. Consequently, we registered a notable 2.7% accuracy enhancement on the ImageNet100 dataset in ViT. Our wavelet patching module, designed for versatility, seamlessly fits into various ViT derivatives without necessitating architecture modifications. This advancement has uplifted the performance of several leading vision transformers by 0.46–4.3%, preserving parameter efficiency without notable FLOPs increment.

The aim of end-to-end sign language translation (SLT) is to interpret continuous sign language (SL) video sequences into coherent natural language sentences without any intermediary annotations, i.e., glosses. However, end-to-end SLT suffers several intractable issues: (i) the temporal correspondence constraint loss problem between SL videos and glosses, and (ii) the weakly supervised sequence labeling problem between long SL videos and sentences. To address these issues, we propose an adaptive video representation enhanced Transformer (AVRET), with three extra modules: adaptive masking (AM), local clip self-attention (LCSA) and adaptive fusion (AF). Specifically, we utilize the first AM module to generate a special mask that adaptively drops out temporally important SL video frame representations to enhance the SL video features. Then, we pass the masked video feature to the Transformer encoder consisting of LCSA and masked self-attention to learn clip-level and continuous video-level feature information. Finally, the output feature of encoder is fused with the temporal feature of AM module via the AF module and use the second AM module to generate more robust feature representations. Besides, we add weakly supervised loss terms to constrain these two AM modules. To promote the Chinese SLT research, we further construct CSL-FocusOn, a Chinese continuous SLT dataset, and share its collection method. It involves many common scenarios, and provides SL sentence annotations and multi-cue images of signers. Our experiments on the CSL-FocusOn, PHOENIX14T, and CSL-Daily datasets show that the proposed method achieves the competitive performance on the end-to-end SLT task without using glosses in training. The code is available at https://github.com/LzDddd/AVRET.

The Segment Anything Model (SAM) is a foundational model that has demonstrated impressive results in the field of natural image segmentation. However, its performance remains suboptimal for medical image segmentation, particularly when delineating lesions with irregular shapes and low contrast. This can be attributed to the significant domain gap between medical images and natural images on which SAM was originally trained. In this paper, we propose an adaptation of SAM specifically tailored for lesion segmentation termed LeSAM. LeSAM first learns medical-specific domain knowledge through an efficient adaptation module and integrates it with the general knowledge obtained from the pre-trained SAM. Subsequently, we leverage this merged knowledge to generate lesion masks using a modified mask decoder implemented as a lightweight U-shaped network design. This modification enables better delineation of lesion boundaries while facilitating ease of training. We conduct comprehensive experiments on various lesion segmentation tasks involving different image modalities such as CT scans, MRI scans, ultrasound images, dermoscopic images, and endoscopic images. Our proposed method achieves superior performance compared to previous state-of-the-art methods in 8 out of 12 lesion segmentation tasks while achieving competitive performance in the remaining 4 datasets. Additionally, ablation studies are conducted to validate the effectiveness of our proposed adaptation modules and modified decoder.

Graph contrastive learning has gained popularity due to its success in self-supervised graph representation learning. Augmented views in contrastive learning greatly determine the quality of the learned representations. Handcrafted data augmentations in previous work require tedious trial-and- errors per dataset, which is time-consuming and resource-intensive. Here, we propose a Graph Contrastive learning framework with Learnable graph Augmentation called GraphCLA. Specifically, learnable graph augmentation trains augmented views by minimizing the mutual information (MI) between the input graphs and their augmented graphs. This paper designs a kernel contrastive loss function based on an end-to-end differentiable graph kernel to learn augmented views. In addition, this paper utilizes a min-max optimization strategy to learn challenging augmented graphs and to learn discriminative representations adversarially. Finally, we compared GraphCLA with state-of-the-art self-supervised learning baselines and experimentally validate the effectiveness of GraphCLA.

Prediction of brain functional activity is of great significance for neuroscience research. The brain functional activities at different regions are highly related, and their relationships can be captured with functional connectivity and structural connectivity. The existing works are challenging to integrate two connectivity information for functional activity prediction. In this paper, we propose a Temporal Cross-Graph Network (TCGN) for predicting brain functional activity, which can comprehensively exploit multi-modal spatial dependence and temporal patterns. In particular, a novel cross-graph convolution module is developed to capture the spatial features of brain structural and functional connectivity. A temporal fusion module is designed to learn the pattern of dynamic functional connectivity to guide the prediction. Specially, a multi-task loss function is proposed to incorporate functional activity and dynamic functional connectivity. Extensive experiments on the Human Connectome Project dataset demonstrate the effectiveness of the proposed framework.

Brain tissue segmentation from magnetic resonance imaging (MRI) is of significant importance for clinical application and cognitive research. The promising deep learning based methods heavily depend on the quality and quantity of training datasets, and also ignore the domain knowledge. To overcome this issue, this paper proposes a novel Iterative Seeded Region Growing (ISRG) approach for brain tissue segmentation with only one reference image. After super-voxel generation and matching, we first select the high confidence seeded regions based on the high similarity between individual brain images. Then, we obtain initial the voxel-wise tissue probabilities with a proposed fully convolutional network (named TPUNet). Thirdly, the seeded regions are updated according to the voxel-wise tissue probabilities. The second and the third steps are iteratively performed until the segmentation labels of the entire image are obtained. The proposed approach is evaluated on IBSR18 dataset and achieves better results compared with other methods.

Graph Neural Networks (GNNs) have achieved remarkable performance in extracting structure-aware node representations for graph signal data. However, existing GNNs overly emphasize the consistency of neighbor nodes in the limited receptive field and severely overlook the wide-field information. In this paper, we propose a novel GCN2Caps that transfers multi-field node representations from graph convolutional network (GCN) to capsule network for wide-field graph learning. Specifically, multi-field GCN is employed to extend the receptive field and extract multi-field features. To perform multi-field interactions, GCN2Caps then explores the inherent relationships between multi-field features and generates the wide-field features through the capsule mechanism. On the basis of the wide-field features, a wide-field min-cut graph constraint is introduced to the loss function to execute complementary constraints on the original graph structure. Extensive experiments on three real-world datasets demonstrate that GCN2Caps significantly outperforms stateof-the-art baselines on semi-supervised node classification and the robustness of GCN2Caps is further validated.

The nature of thick-slice scanning causes severe inter-slice discontinuities of 3D medical images, and the vanilla 2D/3D convolutional neural networks (CNNs) fail to represent sparse inter-slice information and dense intra-slice information in a balanced way, leading to severe underfitting to inter-slice features (for vanilla 2D CNNs) and overfitting to noise from long-range slices (for vanilla 3D CNNs). In this work, a novel mesh network (MNet) is proposed to balance the spatial representation inter axes via learning. 1) Our MNet latently fuses plenty of representation processes by embedding multi-dimensional convolutions deeply into basic modules, making the selections of representation processes flexible, thus balancing representation for sparse inter-slice information and dense intra-slice information adaptively. 2) Our MNet latently fuses multi-dimensional features inside each basic module, simultaneously taking the advantages of 2D (high segmentation accuracy of the easily recognized regions in 2D view) and 3D (high smoothness of 3D organ contour) representations, thus obtaining more accurate modeling for target regions. Comprehensive experiments are performed on four public datasets (CT\&MR), the results consistently demonstrate the proposed MNet outperforms the other methods. The code and datasets are available at: https://github.com/zfdong-code/MNet

Graph neural network (GNN) is popular now to solve the tasks in non-Euclidean space and most of them learn deep embeddings by aggregating the neighboring nodes. However, these methods are prone to some problems such as over-smoothing because of the single-scale perspective field and the nature of low-pass filter. To address these limitations, we introduce diffusion scattering network (DSN) to exploit high-order patterns. With observing the complementary relationship between multi-layer GNN and DSN, we propose Hierarchical Diffusion Scattering Graph Neural Network (HDS-GNN) to efficiently bridge DSN and GNN layer by layer to supplement GNN with multi-scale information and band-pass signals. Our model extracts node-level scattering representations by intercepting the low-pass filtering, and adaptively tunes the different scales to regularize multi-scale information. Then we apply hierarchical representation enhancement to improve GNN with the scattering features. We benchmark our model on nine real-world networks on the transductive semi-supervised node classification task. The experimental results demonstrate the effectiveness of our method.

Epilepsy is one of the most common neurological diseases, which can seriously affect the patient's psychological well-being and quality of life. An accurate and reliable seizure prediction system can generate alarm before epileptic seizures to provide patients and their caregivers with sufficient time to take appropriate action. This study proposes an efficient seizure prediction system based on deep learning in order to anticipate the onset of the seizure as early as possible. Handcrafted features extracted based on the prior knowledge and hidden deep features are complementarily fused through the feature fusion module, and then the hybrid features are fed into the multiplicative long short-term memory (MLSTM) to explore the temporal dependency in EEG signals. A one-dimensional channel attention mechanism is implemented to emphasize the more representative information in the multi-channel output of the MLSTM. Finally, a transfer learning strategy is proposed to transfer the weights of the base model trained on the EEG data of all patients to the target patient model, and the latter is then continuously trained using the EEG data of the target patient. The proposed method achieves an average sensitivity of 95.56% and a false positive rate (FPR) of 0.27/h on the SWEC-ETHZ intracranial EEG data. For the more challenging CHB-MIT scalp EEG database, an average sensitivity of 89.47% and a FPR of 0.34/h are obtained. Experimental results demonstrate that the proposed method has good robustness and generalization ability in both intracranial and scalp EEG signals.

Major depressive disorder (MDD) is a common and severe psychiatric illness marked by loss of interest and low energy, which result in the highest burden of disability among all mental disorders. Clinical MDD diagnosis still utilizes the phenomenological approach of syndrome-based interview, which leads to a high rate of misdiagnosis. Therefore, it is highly imperative to explore effective biomarkers to enable precise personalized diagnosis. There still exist two main challenges due to complexity of MDD and individual differences. On the one hand, discriminative features need to be investigated to better reflect the characteristics of MDD. On the other hand, the performance from shallow and static learning models is still not satisfactory. To overcome these issues, we propose a novel Multi-Stage Graph Fusion Networks (MSGFN) for major depressive disorder diagnosis. At first, functional connectivity is calculated to better characterize interactions between white matter and gray matter. Second, multi-stage features are obtained by a deep subspace learning model, and a number of graphs are constructed under the self-expression constraints at each stage. Finally, a novel graph convolutional fusion module is proposed with graph convolutional operations to integrate features and graph at each stage. Extensive experiments demonstrate the superior performance of the proposed framework. Our source code is available on:https://github.com/LIST-KONG/MSGFN-master.

Tubular structure tracking is a crucial task in the fields of computer vision and medical image analysis. The minimal paths-based approaches have exhibited their strong ability in tracing tubular structures, by which a tubular structure can be naturally modeled as a minimal geodesic path computed with a suitable geodesic metric. However, existing minimal paths-based tracing approaches still suffer from difficulties such as the shortcuts and short branches combination problems, especially when dealing with the images involving complicated tubular tree structures or background. In this paper, we introduce a new minimal paths-based model for minimally interactive tubular structure centerline extraction in conjunction with a perceptual grouping scheme. Basically, we take into account the prescribed tubular trajectories and curvature-penalized geodesic paths to seek suitable shortest paths. The proposed approach can benefit from the local smoothness prior on tubular structures and the global optimality of the used graph-based path searching scheme. Experimental results on both synthetic and real images prove that the proposed model indeed obtains outperformance comparing with the state-of-the-art minimal paths-based tubular structure tracing algorithms.

Deformable medical image registration estimates corresponding deformation to align the regions of interest (ROIs) of two images to a same spatial coordinate system. However, recent unsupervised registration models only have correspondence ability without perception, making misalignment on blurred anatomies and distortion on task-unconcerned backgrounds. Label-constrained (LC) registration models embed the perception ability via labels, but the lack of texture constraints in labels and the expensive labeling costs causes distortion internal ROIs and overfitted perception. We propose the first few-shot deformable medical image registration framework, Perception-Correspondence Registration (PC-Reg), which embeds perception ability to registration models only with few labels, thus greatly improving registration accuracy and reducing distortion. 1) We propose the Perception-Correspondence Decoupling which decouples the perception and correspondence actions of registration to two CNNs. Therefore, independent optimizations and feature representations are available avoiding interference of the correspondence due to the lack of texture constraints. 2) For few-shot learning, we propose Reverse Teaching which aligns labeled and unlabeled images to each other to provide supervision information to the structure and style knowledge in unlabeled images, thus generating additional training data. Therefore, these data will reversely teach our perception CNN more style and structure knowledge, improving its generalization ability. Our experiments on three datasets with only five labels demonstrate that our PC-Reg has competitive registration accuracy and effective distortion-reducing ability. Compared with LC-VoxelMorph( λ = 1), we achieve the 12.5%, 6.3% and 1.0% Reg-DSC improvements on three datasets, revealing our framework with great potential in clinical application.

Accurate and robust cephalometric image analysis plays an essential role in orthodontic diagnosis, treatment assessment and surgical planning. This paper proposes a novel landmark localization method for cephalometric analysis using multiscale image patch-based graph convolutional networks. In detail, image patches with the same size are hierarchically sampled from the Gaussian pyramid to well preserve multiscale context information. We combine local appearance and shape information into spatialized features with an attention module to enrich node representations in graph. The spatial relationships of landmarks are built with the incorporation of three-layer graph convolutional networks, and multiple landmarks are simultaneously updated and moved toward the targets in a cascaded coarse-to-fine process. Quantitative results obtained on publicly available cephalometric X-ray images have exhibited superior performance compared with other state-of-the-art methods in terms of mean radial error and successful detection rate within various precision ranges. Our approach performs significantly better especially in the clinically accepted range of 2 mm and this makes it suitable in cephalometric analysis and orthognathic surgery.

The accurate 3D left ventricular (LV) myocardium segmentation in short-axis (SAX) view of cardiac magnetic resonance (CMR) is challenged by the sparse spatial structure of CMR. The strategy of multi-view CMR fusion can provide fine-grained spatial structure for accurate segmentation. However, the large information misalignment and lack of dense 3D CMR as fusion target in multi-view CMR fusion, and the different spatial resolution between the fusion result and the ground truth in segmentation limit the strategy. In this study, we propose a multi-view spatial-aware adversarial network (MVSGAN). It studies the perception of fine-grained cardiac structure for accurate segmentation by the spatialaware multi-view CMR fusion. It consists of three modules: (1) A residual adversarial fusion (RAF) module takes inter-slices deep correlation and anatomical prior to refine the spatial structures by residual supplement and adversarial optimization. (2) A structural perception-aggregation (SPA) module establishes the spatial correlation between the dense cardiac model and sparse label for accurate CMR LV myocardium segmentation. (3) A joint training strategy utilizes the dense SAX volume as explicit and implicit goals to jointly optimize the framework. The experiments are applied on a public dataset and a clinical dataset to evaluate the performance of MVSGAN. The average Dice and Jaccard score of LV myocardium segmentation obtained by MVSGAN are highest among seven existing state-of-the-art methods, which are up to 0.92 and 0.75. It is concluded that the spatial-aware multi-view CMR fusion can provide meaningful spatial correlation for accurate LV myocardium segmentation.

In this paper, we propose a new minimal path-based framework for minimally interactive tubular structure tracking in conjunction with a perceptual grouping scheme. The minimal path models have shown great advantages in tubular structures tracing. However, they suffer from shortcuts or short branches combination problems especially in the case of tubular network with complicated structures or background. Thus, we utilize the curvature-penalized minimal paths and the prescribed tubular trajectories to seek the desired shortest path. The proposed approach benefits from the local smoothness prior on tubular structures and the global optimality of the graph-based path searching scheme. Experimental results on synthetic and real images prove that the proposed model indeed obtains outperformance to state-of-the-art minimal path-based algorithms.

Receptive field-based segmentation models represent features in receptive fields having weak perception for thin semantics in thin structures segmentation, due to the challenges in small local size and large global variation. High-frequency (HiFe) components have strong thin perception ability and is stable for global variation, but its weak adaptability limits its direct application. We propose a HiFe priori rule which enables the network to adaptively extract and fuse HiFe components, enhancing the thin semantics and making the network naturally prefer thin structures for their segmentation. We further propose High-Frequency Semantics Enhancement Network (HiFeNet) based on our HiFe priori rule, boosting the SOTA methods in thin structures segmentation: 1) Our Deep High Frequency (DHiFe) block learns to extract task-dependent HiFe components and adds them to feature maps, achieving great perception of thin structures. 2) Our Latent Residual Denoising (LRD) block progressively weakens task-independent features via hierarchical residuals and learns to fuse HiFe components back to feature maps, further enhancing the thin semantics and weakening the interference of global variation. Extensive experiments on the retinal vessel [1], [2], [3] and Massachusetts road [4] segmentation datasets show great superiority of our HiFeNet.

Accurate coronary lumen segmentation on coronary-computed tomography angiography (CCTA) images is crucial for quantification of coronary stenosis and the subsequent computation of fractional flow reserve. Many factors including difficulty in labeling coronary lumens, various morphologies in stenotic lesions, thin structures and small volume ratio with respect to the imaging field complicate the task. In this work, we fused the continuity topological information of centerlines which are easily accessible, and proposed a novel weakly supervised model, Examinee-Examiner Network (EE-Net), to overcome the challenges in automatic coronary lumen segmentation. First, the EE-Net was proposed to address the fracture in segmentation caused by stenoses by combining the semantic features of lumens and the geometric constraints of continuous topology obtained from the centerlines. Then, a Centerline Gaussian Mask Module was proposed to deal with the insensitiveness of the network to the centerlines. Subsequently, a weakly supervised learning strategy, Examinee-Examiner Learning, was proposed to handle the weakly supervised situation with few lumen labels by using our EE-Net to guide and constrain the segmentation with customized prior conditions. Finally, a general network layer, Drop Output Layer, was proposed to adapt to the class imbalance by dropping well-segmented regions and weights the classes dynamically. Extensive experiments on two different data sets demonstrated that our EE-Net has good continuity and generalization ability on coronary lumen segmentation task compared with several widely used CNNs such as 3D-UNet. The results revealed our EE-Net with great potential for achieving accurate coronary lumen segmentation in patients with coronary artery disease. Code at http://github.com/qiyaolei/Examinee-Examiner-Network.

Accurate segmentation of uterus, uterine fibroids, and spine from MR images is crucial for high intensity focused ultrasound (HIFU) therapy but remains still difficult to achieve because of 1) the large shape and size variations among individuals, 2) the low contrast between adjacent organs and tissues, and 3) the unknown number of uterine fibroids. To tackle this problem, in this paper, we propose a large kernel Encoder-Decoder Network based on a 2D segmentation model. The use of this large kernel can capture multi-scale contexts by enlarging the valid receptive field. In addition, a deep multiple atrous convolution block is also employed to enlarge the receptive field and extract denser feature maps. Our approach is compared to both conventional and other deep learning methods and the experimental results conducted on a large dataset show its effectiveness.

In this paper, an unsupervised cycle image registration convolutional neural network named CIRNet is developed for 3D medical image registration. Different from most deep learning based registration methods that require known spatial transforms, our proposed method is trained in an unsupervised way and predicts the dense displacement vector field. The CIRNet is composed by two image registration modules which have the same architecture and share the parameters. A cycle identical loss is designed in the CIRNet to provide additional constraints to ensure the accuracy of the predicted dense displacement vector field. The method is evaluated by the registration in 4D (3D+t) cardiac CT and MRI images respectively. Quantitative evaluation results demonstrate that our method performs better than the other two existing image registration algorithms. Especially, compared to the traditional image registration methods, our proposed network can finish 3D image registration in less than one second.

Accounting for nearly 2% of all adults, renal cell carcinomas are sensitive to laparoscopic partial nephrectomy (LPN) which needs an accurate diagnosis and localization before operation. Faced with various intensity distribution, erratic location, irregular shape, etc, the image classification and semantic segmentation on CT scans of renal tumor are challenges. This paper presents a multi-task network, segmentation and classification convolutional neural network (SCNet), for preoperative assessment of renal tumor. Via the combination of two tasks, semantic features are fed to the classification network and classification results give segmentation network feedbacks in return. Besides, a 2-step segmentation strategy is conducted to the segmentation module which improves the result by 2.8%. Our experimental results of classification and segmentation achieve 100% accuracy and 0.882 dice coefficient of tumor region respectively, which are better than the results of a single classification network and segmentation network.

In neuroscience research, brain tissue segmentation from magnetic resonance imaging is of significant importance. A challenging issue is to provide an accurate segmentation due to the tissue heterogeneity, which is caused by noise, bias filed and partial volume effects. To overcome these problems, we propose a novel brain MRI segmentation algorithm, the originality of which stands on the combination of supervoxels with graph convolutional networks. Supervoxels are generated from the 3D MRI image with the help of an improved simple linear iterative clustering algorithm. A graph is then built from these supervoxels through the K nearest neighbor algorithm, before being sent to GCNs for tissues classification. The proposed method is evaluated on the two common datasets- the BrainWeb18 dataset and the Internet Brain Segmentation Repository 18 dataset. Experiments demonstrate the performance of our method and that it is better than well-known state-of-the-art methods such as FMRIB software library, statistical parametric mapping, adaptive graph filter.

The wide applications of X-ray computed tomography (CT) bring low-dose CT (LDCT) into a clinical prerequisite, but reducing the radiation exposure in CT often leads to significantly increased noise and artifacts, which might lower the judgment accuracy of radiologists. In this paper, we put forward a domain progressive 3D residual convolution network (DP-ResNet) for the LDCT imaging procedure that contains three stages: sinogram domain network (SD-net), filtered back projection (FBP), and image domain network (ID-net). Though both are based on the residual network structure, the SD-net and ID-net provide complementary effect on improving the final LDCT quality. The experimental results with both simulated and real projection data show that this domain progressive deep-learning network achieves significantly improved performance by combing the network processing in the two domains.

Renal cancer is one of ten most common cancers in human beings. The laparoscopic partial nephrectomy (LPN) becomes the main therapeutic approach in treating renal cancer. Accurate kidney and tumor segmentation in CT images is a prerequisite step in the surgery planning. However, automatic and accurate kidney and renal tumor segmentation in CT images remains a challenge. In this paper, we propose a new method to perform a precise segmentation of kidney and renal tumor in CT angiography images. This method relies on a three-dimensional (3D) fully convolutional network (FCN) which combines a pyramid pooling module (PPM). The proposed network is implemented as an end-to-end learning system directly on 3D volumetric images. It can make use of the 3D spatial contextual information to improve the segmentation of the kidney as well as the tumor lesion. The experiments conducted on 140 patients show that these target structures can be segmented with a high accuracy. The resulting average dice coefficients obtained for kidney and renal tumor are equal to 0.931 and 0.802 respectively. These values are higher than those obtained from the other two neural networks.

Noise detection accuracy is crucial in suppressing random-valued impulse noise. Both false and miss detections determine the final estimation performance. Deterministic detection methods, which distinctly classify pixels into noisy or uncorrupted pixels, tend to increase the estimation error because some uncorrupted edge points are hard to discriminate from the random-valued impulse noise points. This paper proposes an iterative structure-adaptive fuzzy estimation (SAFE) for random-valued impulse noise suppression. This SAFE method is developed in the framework of Gaussian maximum likelihood estimation. The structure-adaptive fuzziness is reflected by two structure-adaptive metrics based on pixel reliability and patch similarity, respectively. The reliability metric for each pixel (as noise free) is estimated via a novel-minimal-path-based structure propagation to give full consideration of the spatially varying image structures. A robust iteration stopping strategy is also proposed by evaluating the reestimation error of the uncorrupted intensity information. The comparative experimental results show that the proposed structure-adaptive fuzziness can lead to effective restoration. An efficient implementation of this SAFE method is also realized via graphics-processing-unit-based parallelization.

Low-dose computed tomography (LDCT) images are often highly degraded by amplified mottle noise and streak artifacts. Maintaining image quality under low-dose scan protocols is a well-known challenge. Recently, sparse representation-based techniques have been shown to be efficient in improving such CT images. In this paper, we propose a 3D feature constrained reconstruction (3D-FCR) algorithm for LDCT image reconstruction. The feature information used in the 3D-FCR algorithm relies on a 3D feature dictionary constructed from available high quality standard-dose CT sample. The CT voxels and the sparse coefficients are sequentially updated using an alternating minimization scheme. The performance of the 3D-FCR algorithm was assessed through experiments conducted on phantom simulation data and clinical data. A comparison with previously reported solutions was also performed. Qualitative and quantitative results show that the proposed method can lead to a promising improvement of LDCT image quality.

In this paper, we propose a new simple and learning-free deep learning network named MomentsNet, whose convolution layer, nonlinear processing layer and pooling layer are constructed by Moments kernels, binary hashing and block-wise histogram, respectively. Twelve typical moments (including geometrical moment, Zernike moment, Tchebichef moment, etc.) are used to construct the MomentsNet whose recognition performance for binary image is studied. The results reveal that MomentsNet has better recognition performance than its corresponding moments in almost all cases and ZernikeNet achieves the best recognition performance among MomentsNet constructed by twelve moments. ZernikeNet also shows better recognition performance on a binary image database than that of PCANet, which is a learning-based deep learning network.

As a particular case of the finite mixture model, Rayleigh mixture model (RMM) is considered as a useful tool for medical ultrasound (US) image segmentation. However, conventional RMM relies on intensity distribution only and does not take any spatial information into account that leads to misclassification on boundaries and inhomogeneous regions. The authors proposed an improved RMM with neighbour (RMMN) information to solve this problem by introducing neighbourhood information through a mean template. The incorporation of the spatial information made RMMN more robust to noise on the boundaries. The size of the window which incorporates neighbour information was resized adaptively according to the local gradient distribution. They evaluated their model on experiments on synthetic data and real US images used by high‐intensity focused ultrasound therapy. On this data, they demonstrated that the proposed model outperforms several state‐of‐the‐art methods in terms of both segmentation accuracy and computation time.

In this paper, we focus on the "blind" identification of the computed tomography (CT) scanner that has produced a CT image. To do so, we propose a set of noise features derived from the image chain acquisition and which can be used as CT-scanner footprint. Basically, we propose two approaches. The first one aims at identifying a CT scanner based on an original sensor pattern noise (OSPN) that is intrinsic to the X-ray detectors. The second one identifies an acquisition system based on the way this noise is modified by its three-dimensional (3-D) image reconstruction algorithm. As these reconstruction algorithms are manufacturer dependent and kept secret, our features are used as input to train a support vector machine (SVM) based classifier to discriminate acquisition systems. Experiments conducted on images issued from 15 different CT-scanner models of 4 distinct manufacturers demonstrate that our system identifies the origin of one CT image with a detection rate of at least 94% and that it achieves better performance than sensor pattern noise (SPN) based strategy proposed for general public camera devices.

In low dose computed tomography (LDCT) imaging, the data inconsistency of measured noisy projections can significantly deteriorate reconstruction images. To deal with this problem, we propose here a new sinogram restoration approach, the sinogram- discriminative feature representation (S-DFR) method. Different from other sinogram restoration methods, the proposed method works through a 3-D representation-based feature decomposition of the projected attenuation component and the noise component using a well-designed composite dictionary containing atoms with discriminative features. This method can be easily implemented with good robustness in parameter setting. Its comparison to other competing methods through experiments on simulated and real data demonstrated that the S-DFR method offers a sound alternative in LDCT.

Minimal path techniques can efficiently extract geometrically curve-like structures by finding the path with minimal accumulated cost between two given endpoints. Though having found wide practical applications (e.g., line identification, crack detection, and vascular centerline extraction), minimal path techniques suffer from some notable problems. The first one is that they require setting two endpoints for each line to be extracted (endpoint problem). The second one is that the connection might fail when the geodesic distance between the two points is much shorter than the desirable minimal path (shortcut problem). In addition, when connecting two distant points, the minimal path connection might become inefficient as the accumulated cost increases over the propagation and results in leakage into some non-feature regions near the starting point (accumulation problem). To address these problems, this paper proposes an approach termed minimal path propagation with backtracking. We found that the information in the process of backtracking from reached points can be well utilized to overcome the above problems and improve the extraction performance. The whole algorithm is robust to parameter setting and allows a coarse setting of the starting point. Extensive experiments with both simulated and realistic data are performed to validate the performance of the proposed method.

This paper proposes an multilinear discriminant analysis network (MLDANet) for the recognition of multidimensional objects, knows as tensor objects. The MLDANet is a variation of linear discriminant analysis network (LDANet) and principal component analysis network (PCANet), both of which are the recently proposed deep learning algorithms. The MLDANet consists of three parts: 1) The encoder learned by MLDA from tensor data. 2) Features maps obtained from decoder. 3) The use of binary hashing and histogram for feature pooling. A learning algorithm for MLDANet is described. Evaluations on UCF11 database indicate that the proposed MLDANet outperforms the PCANet, LDANet, MPCA+LDA, and MLDA in terms of classification for tensor objects.

In this paper, our aim is to investigate the control of bias accumulation when estimating mutual information from nearest neighbors non-parametric approach with continuously distributed random data. Using a multidimensional Taylor series expansion, a general relationship between the estimation bias and neighborhood size for plug-in entropy estimator is established without any assumption on the data for two different norms. When applied with the maximum norm, our theoretical analysis explains experimental simulation tests drawn in existing literature. In the experiments, two different strategies are tested and compared to estimate mutual information on independent and dependent simulated signals.

In this paper, we present the results of a preliminary work which focuses on identifying the system one Digital Radiography (DR) image has been produced by. To do so, we adapt one solution proposed for digital camera devices where the photo response non-uniformity noise of charge coupled device sensors is used as camera fingerprint. In particular, we show that DR acquisition systems leave a similar Digital Radiography Pattern Noise (DRPN) that can be used as fingerprint. In order to extract this DRPN and due to the nature of DR images, we further propose to take advantage of contourlet filtering. Experiments conducted on images issued from 7 different DR systems show first it is possible to identify with good accuracy the origin of one DR image and, second, that contourlet filtering leads to better detection performance than commonly used approaches based on wavelet or BM3D filtering.

A new divergence measure for rigid image registration is proposed that uses the properties of the Arimoto entropy. This Jensen-Arimoto divergence allows designing a novel registration method by minimizing a dissimilarity measure through the steepest gradient descent optimization method. Preliminary experiments on simulated magnetic resonance images with partial overlap and different degrees of noise have been carried out and a comparison has been conducted with other relevant information theoretic measures such as the normalized mutual information and the cross cumulative residual entropy. The results show that the proposed registration approach has better robustness to noise and can provide better registration accuracy, i.e. a sub pixel accuracy less than 0.1mm and 0.1 degree for translation and rotation. In addition, the calculation time for a 2D rigid registration is improved by approximately 10-20 % compared to the other two methods.

In this study, a new quaternion filter for removal of Gaussian noise in colour images is presented. It is based on the quaternion representation of colour images and the optimisation of a tight bound of the quaternion mean‐square error between the restored colour image and the original one, together with the essential idea of the non‐local means filter. The optimal weights are obtained by using the method of Lagrange multipliers. The authors' quaternion optimal weights non‐local means filter is given by the weighted means of the observed quaternion representation using the optimal weights. Experiments on commonly used images are provided to illustrate the efficiency of the proposed filter.

The rotation invariance of the classical disc-based moments, such as Zernike moments (ZMs), pseudo-ZMs (PZMs), and orthogonal Fourier-Mellin moments (OFMMs), makes them attractive as descriptors for the purpose of recognition tasks. However, less work has been performed for the generalization of these moment functions. In this paper, four general forms are developed to obtain a class of disc-based generalized radial polynomials that are orthogonal over the unit circle. These radial polynomials are scaled to ensure numerical stability, and some useful properties are discussed for potential applications they could be used in. Then, these scaled radial polynomials are used as kernel functions to construct a series of unit discbased generalized orthogonal moments (DGMs). The variation of parameters in DGMs can form various types of orthogonal moments: 1) generalized ZMs; 2) generalized PZMs; and 3) generalized OFMMs. The classical ZMs, PZMs, and OFMMs correspond to a special case of these three generalized moments for which the free parameter α = 0. Each member of this family will share some excellent properties for image representation and recognition tasks, such as orthogonality and rotation invariance. In addition, we have also developed two algorithms, the so-called m-recursive and n-recursive methods for the computation of these proposed radial polynomials to improve the numerical stability. Experimental results show that the proposed methods are superior to the classical disc-based moments in terms of image representation capability and classification accuracy.

Low-dose computed tomography (LDCT) images are often severely degraded by amplified mottle noise and streak artifacts. These artifacts are often hard to suppress without introducing tissue blurring effects. In this paper, we propose to process LDCT images using a novel image-domain algorithm called "artifact suppressed dictionary learning (ASDL)." In this ASDL method, orientation and scale information on artifacts is exploited to train artifact atoms, which are then combined with tissue feature atoms to build three discriminative dictionaries. The streak artifacts are cancelled via a discriminative sparse representation operation based on these dictionaries. Then, a general dictionary learning processing is applied to further reduce the noise and residual artifacts. Qualitative and quantitative evaluations on a large set of abdominal and mediastinum CT images are carried out and the results show that the proposed method can be efficiently applied in most current CT systems.

The gyrator transform has been proposed in optics a few years ago. By using the theory of quaternion numbers, this paper presents the quaternion gyrator transform (QGT). It is shown that the QGT can be computed via the left-side type of quaternion Fourier transforms. The new transform is applied to color image encryption for validation, where the rotation angles are used as encryption keys making it more secure compared to a recent method using discrete quaternion Fourier transforms (DQFTs). Experimental results show that the proposed encryption algorithm for color image performs as well as the DQFTs method in terms of noise robustness, so that it could be a useful tool for color image encryption.

In this letter, a new algorithm for joint diagonalization of a set of matrices by congruence is proposed to compute the nonnegative joint diagonalizer. The nonnegativity constraint is imposed by means of a square change of variables. Then we formulate the high-dimensional optimization problem into several sequential polynomial subproblems using LU matrix factorization. Numerical experiments on simulated matrices emphasize the advantages of the proposed method, especially in the case of degeneracies such as for low SNR values and a small number of matrices. An illustration of blind separation of nuclear magnetic resonance spectroscopy confirms the validity and improvement of the proposed method.

In this paper, we present a medical image integrity verification system to detect and approximate local malevolent image alterations (e.g., removal or addition of lesions) as well as identifying the nature of a global processing an image may have undergone (e.g., lossy compression, filtering, etc.). The proposed integrity analysis process is based on nonsignificant region watermarking with signatures extracted from different pixel blocks of interest, which are compared with the recomputed ones at the verification stage. A set of three signatures is proposed. The first two devoted to detection and modification location are cryptographic hashes and checksums, while the last one is issued from the image moment theory. In this paper, we first show how geometric moments can be used to approximate any local modification by its nearest generalized 2-D Gaussian. We then demonstrate how ratios between original and recomputed geometric moments can be used as image features in a classifier-based strategy in order to determine the nature of a global image processing. Experimental results considering both local and global modifications in MRI and retina images illustrate the overall performances of our approach. With a pixel block signature of about 200 bit long, it is possible to detect, to roughly localize, and to get an idea about the image tamper.

The context of this work is the analysis of depth electroencephalographic signals recorded with depth electrodes during seizures in patients with drug-resistant epilepsy. Usually, different phases are observed during the seizure process and we aim to determine how cerebral structures get involved during these phases, in particular whether some structures can “drive” other ones. To this end, we consider a pair of signals and use transfer entropy which needs beforehand to choose efficiently the size of two conditioning vectors built on the past values of these signals. In this contribution, we extend a partial mutual information based technique, first developed for monochannel prediction models, to the case of two channels. Experimental results on signals generated either by a linear autoregressive model or by a physiology-based model of coupled neuronal populations support the relevance of the proposed approach.

We present a new fast radix-3 algorithm for the computation of the length-Ngeneralized discrete Hartley transform of type-II (GDHT-II), whereN= 3m,m≥ 2. Then we apply this algorithm to the direct computation of length-NGDHT-II coefficients when given three adjacent length-N/3 GDHT-II coefficients. The computational complexity of the proposed method is lower than that of the traditional approach for lengthN≥ 9. The arithmetic operations can be saved from 19% to 29% forN= 3mvarying from 9 to 243 and from 17% to 29% forN= 3×2mvarying from 12 to 384. Furthermore, the new approach can be easily implemented.

This work presents the first method of digital blind forensics within the medical imaging field with the objective to detect whether an image has been modified by some processing (e.g. filtering, lossy compression and so on). It compares two image features: the Histogram statistics of Reorganized Block-based Discrete cosine transform coefficients (HRBD), originally proposed for steganalysis purposes, and the Histogram statistics of Reorganized Block-based Tchebichef moments (HRBT). Both features serve as input of a set of SVM classifiers built in order to discriminate tampered images from original ones as well as to identify the nature of the global modification one image may have undergone. Performance evaluation, conducted in application to different medical image modalities, shows that these image features can help, independently or jointly, to blindly distinguish image processing or modifications with a detection rate greater than 70%. They also underline the complementarity of these features.

This paper presents two new sets of nonseparable discrete orthogonal Charlier and Meixner moments describing the images with noise and that are noise-free. The basis functions used by the proposed nonseparable moments are bivariate Charlier or Meixner polynomials introduced by Tratnik et al. This study discusses the computational aspects of discrete orthogonal Charlier and Meixner polynomials, including the recurrence relations with respect to variable x and order n. The purpose is to avoid large variation in the dynamic range of polynomial values for higher order moments. The implementation of nonseparable Charlier and Meixner moments does not involve any numerical approximation, since the basis function of the proposed moments is orthogonal in the image coordinate space. The performances of Charlier and Meixner moments in describing images were investigated in terms of the image reconstruction error, and the results of the experiments on the noise sensitivity are given.

Phase slope index is a measure which aims at detecting causal relation of interdependence in multivariate time series. One drawback of this approach relies in its incapability to distinguish the direct and indirect relations. So, in order to identify only direct relations, we propose to replace the ordinary coherence function used in the phase slope index with the partial coherence. Furthermore, we consider and compare two estimators of the coherence functions, the first one based on Fourier transform and the second one on an autoregressive model. In order to cope with the difficult issue of bidirectional flow, which cannot be addressed by the coherence based phase slope index, we propose another index based on the directed transfer function. Experimental results support the relevance of the new indices, both based on autoregressive modeling, in multivariate time series.

The derivation of moment invariants has been extensively investigated in the past decades. In this paper, we construct a set of invariants derived from Zernike moments which is simultaneously invariant to similarity transformation and to convolution with circularly symmetric point spread function (PSF). Two main contributions are provided: the theoretical framework for deriving the Zernike moments of a blurred image and the way to construct the combined geometric-blur invariants. The performance of the proposed descriptors is evaluated with various PSFs and similarity transformations. The comparison of the proposed method with the existing ones is also provided in terms of pattern recognition accuracy, template matching and robustness to noise. Experimental results show that the proposed descriptors perform on the overall better.

Geometric distortions are generally simple and effective attacks for many watermarking methods. They can make detection and extraction of the embedded watermark difficult or even impossible by destroying the synchronization between the watermark reader and the embedded watermark. In this paper, we propose a new watermarking approach which allows watermark detection and extraction under affine transformation attacks. The novelty of our approach stands on a set of affine invariants we derived from Legendre moments. Watermark embedding and detection are directly performed on this set of invariants. We also show how these moments can be exploited for estimating the geometric distortion parameters in order to permit watermark extraction. Experimental results show that the proposed watermarking scheme is robust to a wide range of attacks: geometric distortion, filtering, compression, and additive noise.

Prostate segmentation in 3-D transrectal ultrasound images is an important step in the definition of the intra-operative planning of high intensity focused ultrasound (HIFU) therapy. This paper presents two main approaches for the semi-automatic methods based on discrete dynamic contour and optimal surface detection. They operate in 3-D and require a minimal user interaction. They are considered both alone or sequentially combined, with and without postregularization, and applied on anisotropic and isotropic volumes. Their performance, using different metrics, has been evaluated on a set of 28 3-D images by comparison with two expert delineations. For the most efficient algorithm, the symmetric average surface distance was found to be 0.77 mm.

The completeness property of the invariant descriptors, which is of fundamental importance from the theoretical as well as the practical points of views, has been investigated by several research groups. In this paper, we propose a new approach to derive a complete set of pseudo-Zernike moment invariants. We first establish a relationship between the pseudo-Zernike moments of the original image and those of the image having the same shape but distinct orientation and scale. Based on this relationship, a complete set of scale and rotation invariants is derived. Experimental results show that the proposed method has better performance in pattern recognition compared to existing method.

In this paper, we proposed a set of translation and rotation invariants extracted from Tchebichef moments. A set of Tchebichef moment invariants is derived from the relationship between Tchebichef moments of the original image and those of the transformed image. These invariants are then used for symmetric image recognition. Contrarily to the methods based on the complex moments in symmetric image analysis, our method does not need the pre-selection of moment values. Experimental results show that the proposed method achieves better performance compared to the existing methods.

Moments and moment invariants are useful tool in pattern recognition and image analysis. Conventional methods to deal with color images are based on RGB decomposition or graying. In this paper, by using the theory of quaternions, we introduce a set of quaternion Zernike moments (QZMs) for color images in a holistic manner. It is shown that the QZMs can be obtained via the conventional Zernike moments of each channel. We also construct a set of combined invariants to rotation and translation (RT) using the modulus of central QZMs. Experimental results show that the proposed descriptors are more efficient than the existing ones.

In this paper, we introduce a new algorithm to register images with rotation and image blurring. The characteristic of this approach is that a set of Legendre moment invariants are used to establish the correspondence of corner points between the reference image and the distorted image, after these points are extracted by Harris corner detector. Transformation parameters can be estimated from those matched points.

Discrete orthogonal moments have been recently introduced in the field of image analysis. It was shown that they have better image representation capability than the continuous orthogonal moments. One problem concerning the use of moments as feature descriptors is the high computational cost, which may limit their application to the problems where the online computation is required. In this paper, we present a new approach for fast computation of the 2-D Tchebichef moments. By deriving some properties of Tchebichef polynomials, and using the image block representation for binary images and intensity slice representation for grayscale images, a fast algorithm is proposed for computing the moments of binary and grayscale images. The theoretical analysis shows that the computational complexity of the proposed method depends upon the number of blocks of the image, thus, it can speed up the computational efficiency as far as the number of blocks is smaller than the image size.

Processing blurred images is a key problem in many image applications. Existing methods to obtain blur invariants which are invariant with respect to centrally symmetric blur are based on geometric moments or complex moments. In this paper, we propose a new method to construct a set of blur invariants using the orthogonal Legendre moments. Some important properties of Legendre moments for the blurred image are presented and proved. The performance of the proposed descriptors is evaluated with various point-spread functions and different image noises. The comparison of the present approach with previous methods in terms of pattern recognition accuracy is also provided. The experimental results show that the proposed descriptors are more robust to noise and have better discriminative power than the methods based on geometric or complex moments.

A novel algorithm for fast computation of the modulated complex lapped transform (MCLT) with sine windowing function is presented. For the MCLT of length-2Minput data sequence, the proposed algorithm is based on computing a length-2Mtype-II generalized discrete Hartley transform. Comparison with existing algorithms shows that the proposed method achieves the minimal number of arithmetic operations.

In this paper, the objective was to investigate the heart rate variability in two selected groups of premature infants (sepsis vs non-sepsis). We studied the RR interval series not only by linear methods - time domain and frequency domain, but also by non-linear methods - chaos theory and information theory, in order to find the optimal parameters to distinguish sepsis premature infants from non-sepsis ones. The results show that indexes of information theory are useful parameters for the diagnosis of late neonatal infection in premature infants with recurrent apnea-bradycardia.

The modified discrete cosine transform (MDCT) and the inverse MDCT (IMDCT) are two of the most computationally intensive operations in layer III of MPEG audio coding standard. In this letter, we present a radix-3 algorithm for efficiently computing the MDCT and the corresponding IMDCT of a sequence with length N=2times3m. Comparison of the computational complexity with some known algorithms shows that the proposed approach reduces significantly the number of arithmetic operations

We present in this letter an efficient direct method for the computation of a length-N type-II generalized discrete Hartley transform (GDHT) when given two adjacent length-N/2 GDHT coefficients. The computational complexity of the proposed method is lower than that of the traditional approach for length Nges8. The arithmetic operations can be saved from 16% to 24% for N varying from 16 to 64. Furthermore, the new approach can be easily implemented

Due to the inherent ill-posedness of PET reconstruction, the reconstructed images usually have noise and edge artifacts, and regularization techniques are needed to produce reasonable results. We propose a new minimum cross-entropy (MXE) image reconstruction method for PET based on the total variation (TV) norm constraint. The use of TV is due to the fact that it can effectively reduce the noise in 2D images while preserving edges. In addition, a multiple level set method was incorporated into image reconstruction to identify the shape of the radioactive objects. It is important for some special applications where the shape of tumors should be identified. The initial emission rates used by the multiphase level set method were estimated using a discrete reconstruction method. Experimental results show that the proposed method is more effective.

This paper presents an efficient method for automatic segmentation of blood vessels in retinal images. A set of directional basis filters based on dyadic wavelet transform is designed to enhance blood vessels. It attempts to utilize the linear combination of the wavelet transforms to fix on the blood vessels directional information in retinal images. The directional maps are processed by thresholding scheme in order to segment blood vessels from the background. The proposed thresholding approach evaluates 2-D entropies based on the gray level-gradient co-occurrence matrix. The 2-D thresholding vector that maximizes the edge class entropies is selected. The thresholding method utilizes the gray level and gradient information in the enhanced image. The new method promises the simpleness and flexibility in many image enhancement and segmentation applications.

This paper deals with a so-called "intermediate" description, in other words, the formation of high-level primitives in angiographies. The method is based on an attributed string matching technique capable to capture the shape similarities between low-level primitives (i.e., vessel contours and centerlines). After designing a multiparametric cost function, we propose a multiline pairing algorithm. In order to objectively evaluate its performances, results are first provided on simulated data and then on a set of coronarographic images, where it is shown that anatomically coherent entities like vessel segments and branches can be built, "objects" that can be further individually analyzed for clinical purpose.