While Marching Cubes (MC) and Marching Tetrahedra (MTet) are widely adopted in 3D reconstruction pipelines due to their simplicity and efficiency, their differentiable variants remain suboptimal for mesh extraction. This often limits the quality of 3D meshes reconstructed from point clouds or images in learning-based frameworks. In contrast, clipped CVTs offer stronger theoretical guarantees and yield higher-quality meshes. However, the lack of a differentiable formulation has prevented their integration into modern machine learning pipelines. To bridge this gap, we propose DCCVT, a differentiable algorithm that extracts high-quality 3D meshes from noisy signed distance fields (SDFs) using clipped CVTs. We derive a fully differentiable formulation for computing clipped CVTs and demonstrate its integration with deep learning-based SDF estimation to reconstruct accurate 3D meshes from input point clouds. Our experiments with synthetic data demonstrate the superior ability of DCCVT against state-of-theart methods in mesh quality and reconstruction fidelity. https://wylliamcantincharawi.dev/DCCVT.github.io/

Despite recent advances in Open-Vocabulary Semantic Segmentation (OVSS), existing training-free methods face several limitations: use of computationally expensive affinity refinement strategies, ineffective fusion of transformer attention maps due to equal weighting or reliance on fixed-size Gaussian kernels to reinforce local spatial smoothness, enforcing isotropic neighborhoods. We propose a strong baseline for training-free OVSS termed as NERVE (Neighbourhood & Entropy-guided Random-walk for open-Vocabulary sEgmentation), which uniquely integrates global and fine-grained local information, exploiting the neighbourhood structure from the self-attention layer of a stable diffusion model. We also introduce a stochastic random walk for refining the affinity rather than relying on fixed-size Gaussian kernels for local context. This spatial diffusion process encourages propagation across connected and semantically related areas, enabling it to effectively delineate objects with arbitrary shapes. Whereas most existing approaches treat self-attention maps from different transformer heads or layers equally, our method uses entropy-based uncertainty to select the most relevant maps. Notably, our method does not require any conventional post-processing techniques like Conditional Random Fields (CRF) or Pixel-Adaptive Mask Refinement (PAMR). Experiments are performed on 7 popular semantic segmentation benchmarks, yielding an overall state-of-the-art zero-shot segmentation performance, providing an effective approach to open-vocabulary semantic segmentation. Our project page is publicly available at: https://kunal-mahatha.github.io/nerve.page/.

Medical Vision-language models (VLMs) have shown remarkable performances in various medical imaging domains such as histopathology by leveraging pre-trained, contrastive models that exploit visual and textual information. However, histopathology images may exhibit severe domain shifts, such as staining, contamination, blurring, and noise, which may severely degrade the VLM’s downstream performance. In this work, we introduce Histopath-C, a new benchmark with realistic synthetic corruptions designed to mimic real-world distribution shifts observed in digital histopathology. Our framework dynamically applies corruptions to any available dataset and evaluates Test-Time Adaptation (TTA) mechanisms on the fly. We then propose LATTE, a transductive, low-rank adaptation strategy that exploits multiple text templates, mitigating the sensitivity of histopathology VLMs to diverse text inputs. Our approach outperforms state-of-the-art TTA methods originally designed for natural images across a breadth of histopathology datasets, demonstrating the effectiveness of our proposed design for robust adaptation in histopathology images. Code and data are available at https://github.com/Mehrdad-Noori/Histopath-C.

We analyze Layer Normalization (LN) from a domain (batch) perspective and explain why BatchNorm-style test-time fixes often fail on Transformer backbones. As feature dimension and batch size grow, the per-feature batch marginals after LN’s pre-affine step concentrate at mean ≈ 0 and variance ≈ 1, making cross-batch re-standardization unnecessary and often harmful. This yields a simple rule: keep the pre-affine LN intact and adjust only the post-affine mean and gain. We instantiate this with LN-TTA, a backpropagation-free and source-free, test-time adaptation that performs a single forward pass and uniformly reparameterizes each LN layer. On three corrupted 3D point-cloud suites (ScanObjectNN-C, ModelNet40-C, ShapeNet-C), LN-TTA improves over Source-Only by +12.35, +15.58, and +3.03 points, surpasses backpropagation baselines (e.g., TENT), and sustains up to 93 samples/s, on average 39× faster and 5× more memory-efficient than the next-best backprop-free method. Code is available at: github.com/MosyMosy/LN_TTA.

The accurate segmentation of brain tumors plays an important role in clinical diagnosis and treatment. Multimodal magnetic resonance imaging (MRI) can provide rich and complementary information for accurate brain tumor segmentation. However, the common problems of incomplete modalities and small samples in clinical practice seriously affect the performance of multimodal segmentation. In this work, we design a new framework, named M$^{2}$SegMamba, using Mamba and Masked Autoencoder networks for both supervised and self-supervised learning, aimed at handling small sample brain tumor segmentation under various incomplete multimodality settings. We construct a masking strategy suitable for multimodal brain tumors to precisely extract image features, which serves as the foundation for image segmentation. By fully leveraging the capabilities of the Mamba network, we design a multi-traversal method to facilitate the interaction between inter-modal and cross-modal image features. Meanwhile, the introduction of TSmamba in skipping connections efficiently integrates multimodal features. Auxiliary regularizers are introduced in both the encoder and decoder to further enhance the model's robustness to incomplete modalities. We conducted experiments on the BraTS 2018 and BraTS 2020 datasets, and the results demonstrate that our method outperforms state-of-the-art brain tumor segmentation methods on most subsets of missing modalities.

Domain adaptation methods aim to bridge the gap between datasets by enabling knowledge transfer across domains, reducing the need for additional expert annotations. However, many approaches struggle with reliability in the target domain, an issue particularly critical in medical image segmentation, where accuracy and anatomical validity are essential. This challenge is further exacerbated in spatio-temporal data, where the lack of temporal consistency can significantly degrade segmentation quality, and particularly in echocardiography, where the presence of artifacts and noise can further hinder segmentation performance. To address these issues, we present RL4Seg3D, an unsupervised domain adaptation framework for 2D + time echocardiography segmentation. RL4Seg3D integrates novel reward functions and a fusion scheme to enhance key landmark precision in its segmentations while processing full-sized input videos. By leveraging reinforcement learning for image segmentation, our approach improves accuracy, anatomical validity, and temporal consistency while also providing, as a beneficial side effect, a robust uncertainty estimator, which can be used at test time to further enhance segmentation performance. We demonstrate the effectiveness of our framework on over 30,000 echocardiographic videos, showing that it outperforms standard domain adaptation techniques without the need for any labels on the target domain. Code is available at https://github.com/arnaudjudge/RL4Seg3D.

State Space Models (SSMs) have shown significant promise in Natural Language Processing (NLP) and, more recently, computer vision. This paper introduces a new methodology that leverages Mamba and Masked Autoencoder (MAE) networks for point-cloud data in both supervised and self-supervised learning. We propose three key contributions to enhance Mamba’s capability in processing complex point-cloud structures. First, we exploit the spectrum of a graph Laplacian to capture patch connectivity, defining an isometry-invariant traversal order that is robust to viewpoints and captures shape manifolds better than traditional 3D grid-based traversals. Second, we adapt segmentation via a recursive patch partitioning strategy informed by Laplacian spectral components, allowing finer integration and segment analysis. Third, we address token placement in MAE for Mamba by restoring tokens to their original positions, which preserves essential order and improves learning. Extensive experiments demonstrate the improvements that our approach brings over state-of-the-art baselines in classification, segmentation, and few-shot tasks. The implementation is available at: https://github.com/AliBahri94/SI-Mamba.git.

Recent advances in diffusion models have spurred research into their application for Reconstruction-based unsupervised anomaly detection. However, these methods may struggle with maintaining structural integrity and recovering the anomaly-free content of abnormal regions, especially in multi-class scenarios. Furthermore, diffusion models are inherently designed to generate images from pure noise and struggle to selectively alter anomalous regions of an image while preserving normal ones. This leads to potential degradation of normal regions during reconstruction, hampering the effectiveness of anomaly detection. This paper introduces a reformulation of the standard diffusion model geared toward selective region alteration, allowing the accurate identification of anomalies. By modeling anomalies as noise in the latent space, our proposed Deviation correction diffusion (DeCo-Diff) model preserves the normal regions and encourages transformations exclusively on anomalous areas. This selective approach enhances the reconstruction quality, facilitating effective unsupervised detection and localization of anomaly regions. Comprehensive evaluations demonstrate the superiority of our method in accurately identifying and localizing anomalies in complex images, with pixel-level AUPRC improvements of 11-14% over state-of-the-art models on well-known anomaly detection datasets. The code is available at https://github.com/farzad-bz/DeCo-Diff

State Space Models (SSMs) have recently emerged as an alternative to Vision Transformers (ViTs) due to their unique ability of modeling global relationships with linear complexity. SSMs are specifically designed to capture spatially proximate relationships of image patches. However, they fail to identify relationships between conceptually related yet not adjacent patches. This limitation arises from the non-causal nature of image data, which lacks inherent directional relationships. Additionally, current vision-based SSMs are highly sensitive to transformations such as rotation. Their predefined scanning directions depend on the original image orientation, which can cause the model to produce inconsistent patch-processing sequences after rotation. To address these limitations, we introduce Spectral VMamba, a novel approach that effectively captures the global structure within an image by leveraging spectral information derived from the graph Laplacian of image patches. Through spectral decomposition, our approach encodes patch relationships independently of image orientation, achieving patch traversal rotation invariance with our Rotational Feature Normalizer (RFN) module. Our experiments on classification tasks show that Spectral VMamba outperforms the leading SSM models in vision, such as VMamba, while maintaining invariance to rotations and a providing a similar runtime efficiency. The implementation is available at: https://github.com/Sahardastani/spectral_vmamba.git.

Test-time adaptation (TTA) is crucial for mitigating performance degradation caused by distribution shifts in 3D point cloud classification. In this work, we introduce Token Purging (PG), a novel backpropagation-free approach that removes tokens highly affected by domain shifts before they reach attention layers. Unlike existing TTA methods, PG operates at the token level, ensuring robust adaptation without iterative updates. We propose two variants: PG-SP, which leverages source statistics, and PG-SF, a fully source-free version relying on CLS-token-driven adaptation. Extensive evaluations on ModelNet40-C, ShapeNet-C, and ScanObjectNN-C demonstrate that PG-SP achieves an average of +10.3\% higher accuracy than state-of-the-art backpropagation-free methods, while PG-SF sets new benchmarks for source-free adaptation. Moreover, PG is 12.4 times faster and 5.5 times more memory efficient than our baseline, making it suitable for real-world deployment. Code is available at \hyperlink{https://github.com/MosyMosy/Purge-Gate}{https://github.com/MosyMosy/Purge-Gate}

Test-Time Training has emerged as a promising solution to address distribution shifts in 3D point cloud classification. However, existing methods often rely on computationally expensive backpropagation during adaptation, limiting their applicability in real-world, time-sensitive scenarios. In this paper, we introduce SMART-PC, a skeleton-based framework that enhances resilience to corruptions by leveraging the geometric structure of 3D point clouds. During pre-training, our method predicts skeletal representations, enabling the model to extract robust and meaningful geometric features that are less sensitive to corruptions, thereby improving adaptability to test-time distribution shifts. Unlike prior approaches, SMART-PC achieves real-time adaptation by eliminating backpropagation and updating only BatchNorm statistics, resulting in a lightweight and efficient framework capable of achieving high frame-per-second rates while maintaining superior classification performance. Extensive experiments on benchmark datasets, including ModelNet40-C, ShapeNet-C, and ScanObjectNN-C, demonstrate that SMART-PC achieves state-of-the-art results, outperforming existing methods such as MATE in terms of both accuracy and computational efficiency. The implementation is available at: \url{https://github.com/AliBahri94/SMART-PC}.

State Space Models (SSMs) have emerged as efficient alternatives to Vision Transformers (ViTs), with VMamba standing out as a pioneering architecture designed for vision tasks. However, their generalization performance degrades significantly under distribution shifts. To address this limitation, we propose TRUST (Test-Time Refinement using Uncertainty-Guided SSM Traverses), a novel test-time adaptation (TTA) method that leverages diverse traversal permutations to generate multiple causal perspectives of the input image. Model predictions serve as pseudo-labels to guide updates of the Mamba-specific parameters, and the adapted weights are averaged to integrate the learned information across traversal scans. Altogether, TRUST is the first approach that explicitly leverages the unique architectural properties of SSMs for adaptation. Experiments on seven benchmarks show that TRUST consistently improves robustness and outperforms existing TTA methods.

Vision-language models (VLMs) like CLIP exhibit strong zero-shot capabilities but often fail to generalize under distribution shifts. Test-time adaptation (TTA) allows models to update at inference time without labeled data, typically via entropy minimization. However, this objective is fundamentally misaligned with the contrastive image-text training of VLMs, limiting adaptation performance and introducing failure modes such as pseudo-label drift and class collapse. We propose CLIPTTA, a new gradient-based TTA method for vision-language models that leverages a soft contrastive loss aligned with CLIP’s pre-training objective. We provide a theoretical analysis of CLIPTTA’s gradients, showing how its batch-aware design mitigates the risk of collapse. We further extend CLIPTTA to the open-set setting, where both in-distribution (ID) and out-of-distribution (OOD) samples are encountered, using an Outlier Contrastive Exposure (OCE) loss to improve OOD detection. Evaluated on 75 datasets spanning diverse distribution shifts, CLIPTTA consistently outperforms entropy-based objectives and is highly competitive with state-of-the-art TTA methods, outperforming them on a large number of datasets and exhibiting more stable performance across diverse shifts.

Progress in a research field can be hard to assess, in particular when many concurrent methods are proposed in a short period of time. This is the case in digital pathology, where many foundation models have been released recently to serve as feature extractors for tile-level images, being used in a variety of downstream tasks, both for tile- and slide-level problems. Benchmarking available methods then becomes paramount to get a clearer view of the research landscape. In particular, in critical domains such as healthcare, a benchmark should not only focus on evaluating downstream performance, but also provide insights about the main differences between methods, and importantly, further consider uncertainty and robustness to ensure a reliable usage of proposed models. For these reasons, we introduce THUNDER, a tile-level benchmark for digital pathology foundation models, allowing for efficient comparison of many models on diverse datasets with a series of downstream tasks, studying their feature spaces and assessing the robustness and uncertainty of predictions informed by their embeddings. THUNDER is a fast, easy-to-use, dynamic benchmark that can already support a large variety of state-of-the-art foundation, as well as local user-defined models for direct tile-based comparison. In this paper, we provide a comprehensive comparison of 23 foundation models on 16 different datasets covering diverse tasks, feature analysis, and robustness. The code for THUNDER is publicly available at https://github.com/MICS-Lab/thunder.

Recently, test-time adaptation has attracted wide interest in the context of vision-language models for image classification. However, to the best of our knowledge, the problem is completely overlooked in dense prediction tasks such as Open-Vocabulary Semantic Segmentation (OVSS). In response, we propose a novel TTA method tailored to adapting VLMs for segmentation during test time. Unlike TTA methods for image classification, our Multi-Level and Multi-Prompt (MLMP) entropy minimization integrates features from intermediate vision-encoder layers and is performed with different text-prompt templates at both the global CLS token and local pixel-wise levels. Our approach could be used as plug-and-play for any segmentation network, does not require additional training data or labels, and remains effective even with a single test sample. Furthermore, we introduce a comprehensive OVSS TTA benchmark suite, which integrates a rigorous evaluation protocol, nine segmentation datasets, 15 common synthetic corruptions, and additional real and rendered domain shifts, with a total of 87 distinct test scenarios, establishing a standardized and comprehensive testbed for future TTA research in open-vocabulary segmentation. Our experiments on this suite demonstrate that our segmentation-tailored method consistently delivers significant gains over direct adoption of TTA classification baselines. Code and data are available at https://github.com/dosowiechi/MLMP.

Domain Generalization techniques aim to enhance model robustness by simulating novel data distributions during training, typically through various augmentation or stylization strategies. However, these methods frequently suffer from limited control over the diversity of generated images and lack assurance that these images span distinct distributions. To address these challenges, we propose FDS, Feedback-guided Domain Synthesis, a novel strategy that employs diffusion models to synthesize novel, pseudo-domains by training a single model on all source domains and performing domain mixing based on learned features. By incorporating images that pose classification challenges to models trained on original samples, alongside the original dataset, we ensure the generation of a training set that spans a broad distribution spectrum. Our comprehensive evaluations demonstrate that this methodology sets new benchmarks in domain generalization performance across a range of challenging datasets, effectively managing diverse types of domain shifts. The code can be found at: https://github.com/Mehrdad-Noori/FDS

Test-Time Adaptation (TTA) addresses distribution shifts during testing by adapting a pretrained model without access to source data. In this work, we propose a novel TTA approach for 3D point cloud classification, combining sampling variation with weight averaging. Our method leverages Farthest Point Sampling (FPS) and K-Nearest Neighbors (KNN) to create multiple point cloud representations, adapting the model for each variation using the TENT algorithm. The final model parameters are obtained by averaging the adapted weights, leading to improved robustness against distribution shifts. Extensive experiments on ModelNet40-C, ShapeNet-C, and ScanObjectNN-C datasets, with different backbones (Point-MAE, Point-Net, DGCNN), demonstrate that our approach consistently outperforms existing methods while maintaining minimal resource overhead. The proposed method effectively enhances model generalization and stability in challenging real-world conditions. The implementation is available at: https://github.com/AliBahri94/SVWA_TTA.git.

The remarkable progress in deep learning (DL) show-cases outstanding results in various computer vision tasks. However, adaptation to real-time variations in data distributions remains an important challenge. Test- Time Training (TTT) was proposed as an effective solution to this issue, which increases the generalization ability of trained models by adding an auxiliary task at train time and then using its loss at test time to adapt the model. Inspired by the recent achievements of contrastive representation learning in unsupervised tasks, we propose ReC-TTT, a test-time training technique that can adapt a DL model to new un-seen domains by generating discriminative views of the input data. ReC- Ttt uses cross-reconstruction as an auxiliary task between a frozen encoder and two trainable en-coders, taking advantage of a single shared decoder. This enables, at test time, to adapt the encoders to extract features that will be correctly reconstructed by the decoder that, in this phase, is frozen on the source domain. Experimental results show that ReC- Ttt achieves better re-sults than other state-of-the-art techniques in most domain shift classification challenges. The code is available at: https://github.com/warpcut/ReC-TTT

Pre-trained vision-language models (VLMs), exemplified by CLIP, demonstrate remarkable adaptability across zero-shot classification tasks without additional training. However, their performance diminishes in the presence of domain shifts. In this study, we introduce CLIP Adaptation duRing Test-Time (CLIPArTT), a fully test-time adaptation (TTA) approach for CLIP, which involves automatic text prompts construction during inference for their use as text supervision. Our method employs a unique, minimally invasive text prompt tuning process, wherein multiple predicted classes are aggregated into a single new text prompt, used as pseudo label to re-classify inputs in a transductive manner. Additionally, we pioneer the standardization of TTA benchmarks (e.g., TENT) in the realm of VLMs. Our findings demonstrate that, without requiring additional transformations nor new trainable modules, CLIPArTT enhances performance dynamically across non-corrupted datasets such as CIFAR-100, corrupted datasets like CIFAR-100-C and ImageNet-C, alongside synthetic datasets such as VisDA-C. This research underscores the potential for improving VLMs' adaptability through novel test-time strategies, offering insights for robust performance across varied datasets and environments. The code can be found at: https://github.com/dosowiechi/CLIPArTT.git

Despite the remarkable performance of vision language models (VLMs), such as contrastive language image pretraining (CLIP), the large size of these models is a considerable obstacle to their use in federated learning (FL) systems where the parameters of local client models need to be transferred to a global server for aggregation. Another challenge in FL is the heterogeneity of data from different clients, which affects the generalization performance of the solution. In addition, natural pretrained VLMs exhibit poor generalization ability in the medical datasets, suggests there exists a domain gap. To solve these issues, we introduce a novel method for the federated adversarial adaptation (FAA) of CLIP. Our method, named FAA-CLIP, handles the large communication costs of CLIP using a lightweight feature adaptation module (FAM) for aggregation, effectively adapting this VLM to each client’s data while greatly reducing the number of parameters to transfer. By keeping CLIP frozen and only updating the FAM parameters, our method is also computationally efficient. Unlike existing approaches, our FAA-CLIP method directly addresses the problem of domain shifts across clients via a domain adaptation (DA) module. This module employs a domain classifier to predict if a given sample is from the local client or the global server, allowing the model to learn domain-invariant representations. Extensive experiments on six different datasets containing both natural and medical images demonstrate that FAA-CLIP can generalize well on both natural and medical datasets compared to recent FL approaches. Our codes are available athttps://github.com/AIPMLab/FAA-CLIP.

Lack of standardization and various intrinsic parameters for magnetic resonance (MR) image acquisition results in heterogeneous images across different sites and devices, which adversely affects the generalization of deep neural networks. To alleviate this issue, this work proposes a novel unsupervised harmonization framework that leverages normalizing flows to align MR images, thereby emulating the distribution of a source domain. The proposed strategy comprises three key steps. Initially, a normalizing flow network is trained to capture the distribution characteristics of the source domain. Then, we train a shallow harmonizer network to reconstruct images from the source domain via their augmented counterparts. Finally, during inference, the harmonizer network is updated to ensure that the output images conform to the learned source domain distribution, as modeled by the normalizing flow network. Our approach, which is unsupervised, source-free, and task-agnostic is assessed in the context of both adults and neonatal cross-domain brain MRI segmentation, as well as neonatal brain age estimation, demonstrating its generalizability across tasks and population demographics. The results underscore its superior performance compared to existing methodologies. The code is available at https://github.com/farzad-bz/Harmonizing-Flows.

Despite their exceptional performance in vision tasks, deep learning models often struggle when faced with domain shifts during testing. Test-Time Training (TTT) methods have recently gained popularity by their ability to enhance the robustness of models through the addition of an auxiliary objective that is jointly optimized with the main task. Being strictly unsupervised, this auxiliary objective is used at test time to adapt the model without any access to labels. In this work, we propose Noise-Contrastive TestTime Training (NC-TTT), a novel unsupervised TTT technique based on the discrimination of noisy feature maps. By learning to classify noisy views of projected feature maps, and then adapting the model accordingly on new domains, classification performance can be recovered by an important margin. Experiments on several popular testtime adaptation baselines demonstrate the advantages of our method compared to recent approaches for this task. The code can be found at: https://github.com/GustavoVargasHakim/NCTTT.git

Vision-Language Models (VLMs) such as CLIP have yielded unprecedented performances for zero-shot image classification, yet their generalization capability may still be seriously challenged when confronted to domain shifts. In response, we present Weight Average Test-Time Adaptation (WATT) of CLIP, a new approach facilitating full test-time adaptation (TTA) of this VLM. Our method employs a diverse set of templates for text prompts, augmenting the existing framework of CLIP. Predictions are utilized as pseudo labels for model updates, followed by weight averaging to consolidate the learned information globally. Furthermore, we introduce a text ensemble strategy, enhancing the overall test performance by aggregating diverse textual cues. Our findings underscore the effectiveness of WATT across diverse datasets, including CIFAR-10-C, CIFAR-10.1, CIFAR-100-C, VisDA-C, and several other challenging datasets, effectively covering a wide range of domain shifts. Notably, these enhancements are achieved without the need for additional model transformations or trainable modules. Moreover, compared to other TTA methods, our approach can operate effectively with just a single image. The code is available at: https://github.com/Mehrdad-Noori/WATT.

Despite the recent progress in incremental learning, addressing catastrophic forgetting under distributional drift is still an open and important problem. Indeed, while state-of-the-art domain incremental learning (DIL) methods perform satisfactorily within known domains, their performance largely degrades in the presence of novel domains. This limitation hampers their generalizability, and restricts their scalability to more realistic settings where train and test data are drawn from different distributions. To address these limitations, we present a novel DIL approach based on a mixture of prompt-tuned CLIP models (MoPCLIP), which generalizes the paradigm of S-Prompting to handle both in-distribution and out-of-distribution data at inference. In particular, at the training stage we model the features distribution of every class in each domain, learning individual text and visual prompts to adapt to a given domain. At inference, the learned distributions allow us to identify whether a given test sample belongs to a known domain, selecting the correct prompt for the classification task, or from an unseen domain, leveraging a mixture of the prompt-tuned CLIP models. Our empirical evaluation reveals the poor performance of existing DIL methods under domain shift, and suggests that the proposed MoP-CLIP performs competitively in the standard DIL settings while outperforming state-of-the-art methods in OOD scenarios. These results demonstrate the superiority of MoP-CLIP, offering a robust and general solution to the problem of domain incremental learning.

Generalizing to out-of-distribution (OOD) data is a challenging task for existing deep learning approaches. This problem largely comes from the common but often incorrect assumption of statistical learning algorithms that the source and target data come from the same i.i.d. distribution. To tackle the limited variability of domains available during training, as well as domain shifts at test time, numerous approaches for domain generalization have focused on generating samples from new domains. Recent studies on this topic suggest that feature statistics from instances of different domains can be mixed to simulate synthesized images from a novel domain. While this simple idea achieves state-of-art results on various domain generalization benchmarks, it ignores structural information which is key to transferring knowledge across different domains. In this paper, we leverage the ability of humans to recognize objects using solely their structural information (prominent region contours) to design a Structural-Aware Feature Stylization method for domain generalization. Our method improves feature stylization based on mixing instance statistics by enforcing structural consistency across the different style-augmented samples. This is achieved via a multi-task learning model which classifies original and augmented images while also reconstructing their edges in a secondary task. The edge reconstruction task helps the network preserve image structure during feature stylization, while also acting as a regularizer for the classification task. Through quantitative comparisons, we verify the effectiveness of our method upon existing state-of-the-art methods on PACS, VLCS, OfficeHome, DomainNet and Digits-DG. The implementation is available at this repository.

Due to the fast advancement of artificial intelligence (AI), centralized-based models have become critical for healthcare tasks like in medical image analysis and human behavior recognition. Although these models exhibit suitable performance, they are frequently constrained by privacy concerns. To attenuate this, a centralized learning strategy cannot be used in cases where there is a risk of data privacy breach, particularly in healthcare centers. Federated learning (FL) is a technique that allows for training a global model without sharing data by training distributed local models and aggregating them. By implementing FL throughout the training process, we can obtain a model with comparable generalization abilities to centralized learning while maintaining data privacy. This survey provides an introduction to the fundamental concepts and categories of FL, highlights the limitations of the centralized healthcare model, and discusses how FL can address these constraints. We also provide a detailed overview of the healthcare applications using FL models, along with commonly used evaluation metrics and public data sets. In this context, we have implemented a case study to demonstrate how FL can be applied in the healthcare field. Furthermore, we outline the key challenges and future trends in FL.

Despite the remarkable progress in semi-supervised medical image segmentation methods based on deep learning, their application to real-life clinical scenarios still faces considerable challenges. For example, insufficient labeled data often makes it difficult for networks to capture the complexity and variability of the anatomical regions to be segmented. To address these problems, we design a new semi-supervised segmentation framework that aspires to produce anatomically plausible predictions. Our framework comprises two parallel networks: shape-agnostic and shape-aware networks. These networks learn from each other, enabling effective utilization of unlabeled data. Our shape-aware network implicitly introduces shape guidance to capture shape fine-grained information. Meanwhile, shape-agnostic networks employ uncertainty estimation to further obtain reliable pseudo-labels for the counterpart. We also employ a cross-style consistency strategy to enhance the network's utilization of unlabeled data. It enriches the dataset to prevent overfitting and further eases the coupling of the two networks that learn from each other. Our proposed architecture also incorporates a novel loss term that facilitates the learning of the local context of segmentation by the network, thereby enhancing the overall accuracy of prediction. Experiments on three different datasets of medical images show that our method outperforms many excellent semi-supervised segmentation methods and outperforms them in perceiving shape. The code can be seen at https://github.com/igip-liu/SLC-Net.

Deep Learning models have shown remarkable performance in a broad range of vision tasks. However, they are often vulnerable to domain shifts at test-time. Test-time training (TTT) methods have been developed in an attempt to mitigate these vulnerabilities, where a secondary task is solved at training time, simultaneously with the main task, to be later used as an self-supervised proxy task at test-time. In this work, we propose a novel unsupervised TTT technique based on the maximization of Mutual Information between multi-scale feature maps and a discrete latent representation, which can be integrated to the standard training as an auxiliary clustering task. Experimental results demonstrate competitive classification performance on different popular test-time adaptation benchmarks. The code can be found at: https://github.com/dosowiechi/ClusT3.git

Systems for person re-identification (ReID) can achieve a high accuracy when trained on large fully-labeled image datasets. However, the domain shift typically associated with diverse operational capture conditions (e.g., camera viewpoints and lighting) may translate to a significant decline in performance. This paper focuses on unsupervised domain adaptation (UDA) for video-based ReID – a relevant scenario that is less explored in the literature. In this scenario, the ReID model must adapt to a complex target domain defined by a network of diverse video cameras based on track-let information. State-of-art methods cluster unlabeled target data, yet domain shifts across target cameras (sub-domains) can lead to poor initialization of clustering methods that propagates noise across epochs, thus preventing the ReID model to accurately associate samples of same identity. In this paper, an UDA method is introduced for video person ReID that leverages knowledge on video tracklets, and on the distribution of frames captured over target cameras to improve the performance of CNN backbones trained using pseudo-labels. Our method relies on an adversarial approach, where a camera-discriminator network is introduced to extract discriminant camera-independent representations, facilitating the subsequent clustering. In addition, a weighted contrastive loss is proposed to leverage the confidence of clusters, and mitigate the risk of incorrect identity associations. Experimental results obtained on three challenging video-based person ReID datasets – PRID2011, iLIDS-VID, and MARS – indicate that our proposed method can outperform related state-of-the-art methods. Our code is available at: https://github.com/dmekhazni/CAWCL-ReID

A major problem of deep neural networks for image classification is their vulnerability to domain changes at test-time. Recent methods have proposed to address this problem with test-time training (TTT), where a two-branch model is trained to learn a main classification task and also a self-supervised task used to perform test-time adaptation. However, these techniques require defining a proxy task specific to the target application. To tackle this limitation, we propose TTTFlow: a Y-shaped architecture using an unsupervised head based on Normalizing Flows to learn the nor-mal distribution of latent features and detect domain shifts in test examples. At inference, keeping the unsupervised head fixed, we adapt the model to domain-shifted examples by maximizing the log likelihood of the Normalizing Flow. Our results show that our method can significantly improve the accuracy with respect to previous works.

Deep learning methods have shown outstanding potential in dermatology for skin lesion detection and identification. However, they usually require annotations beforehand and can only classify lesion classes seen in the training set. Moreover, large-scale, open-sourced medical datasets normally have far fewer annotated classes than in real life, further aggravating the problem. This paper proposes a novel method called DNF-OOD, which applies a non-parametric deep forest-based approach to the problem of out-of-distribution (OOD) detection. By leveraging a maximum probabilistic routing strategy and over-confidence penalty term, the proposed method can achieve better performance on the task of detecting OOD skin lesion images, which is challenging due to the large intra-class variability in such images. We evaluate our OOD detection method on images from two large, publicly-available skin lesion datasets, ISIC2019 and DermNet, and compare it against recently-proposed approaches. Results demonstrate the potential of our DNF-OOD framework for detecting OOD skin images.

Deep learning models for semi-supervised medical image segmentation have achieved unprecedented performance for a wide range of tasks. Despite their high accuracy, these models may however yield predictions that are considered anatomically impossible by clinicians. Moreover, incorporating complex anatomical constraints into standard deep learning frameworks remains challenging due to their non-differentiable nature. To address these limitations, we propose a Constrained Adversarial Training (CAT) method that learns how to produce anatomically plausible segmentations. Unlike approaches focusing solely on accuracy measures like Dice, our method considers complex anatomical constraints like connectivity, convexity, and symmetry which cannot be easily modeled in a loss function. The problem of non-differentiable constraints is solved using a Reinforce algorithm which enables to obtain a gradient for violated constraints. To generate constraint-violating examples on the fly, and thereby obtain useful gradients, our method adopts an adversarial training strategy which modifies training images to maximize the constraint loss, and then updates the network to be robust to these adversarial examples. The proposed method offers a generic and efficient way to add complex segmentation constraints on top of any segmentation network. Experiments on synthetic data and four clinically-relevant datasets demonstrate the effectiveness of our method in terms of segmentation accuracy and anatomical plausibility.

We present an unsupervised domain adaptation method for image segmentation which aligns high-order statistics, computed for the source and target domains, encoding domain-invariant spatial relationships between segmentation classes. Our method first estimates the joint distribution of predictions for pairs of pixels whose relative position corresponds to a given spatial displacement. Domain adaptation is then achieved by aligning the joint distributions of source and target images, computed for a set of displacements. Two enhancements of this method are proposed. The first one uses an efficient multi-scale strategy that enables capturing long-range relationships in the statistics. The second one extends the joint distribution alignment loss to features in intermediate layers of the network by computing their cross-correlation. We test our method on the task of unpaired multi-modal cardiac segmentation using the Multi-Modality Whole Heart Segmentation Challenge dataset and prostate segmentation task where images from two datasets are taken as data in different domains. Our results show the advantages of our method compared to recent approaches for cross-domain image segmentation. Code is available at https://github.com/WangPing521/Domain_adaptation_shape_prior.

Batch normalization is a staple of computer vision models, including those employed in few-shot learning. Batch nor-malization layers in convolutional neural networks are composed of a normalization step, followed by a shift and scale of these normalized features applied via the per-channel trainable affine parameters$\gamma$and$\beta$. These affine param-eters were introduced to maintain the expressive powers of the model following normalization. While this hypothesis holds true for classification within the same domain, this work illustrates that these parameters are detrimen-tal to downstream performance on common few-shot trans-fer tasks. This effect is studied with multiple methods on well-known benchmarks such as few-shot classification on minilmageNet, cross-domain few-shot learning (CD-FSL) and META-DATASET. Experiments reveal consistent performance improvements on CNNs with affine unaccompanied batch normalization layers; particularly in large domain-shift few-shot transfer settings. As opposed to common practices in few-shot transfer learning where the affine pa-rameters are fixed during the adaptation phase, we show fine-tuning them can lead to improved performance.

Brain surface analysis is essential to neuroscience, however, the complex geometry of the brain cortex hinders computational methods for this task. The difficulty arises from a discrepancy between 3D imaging data, which is represented in Euclidean space, and the non-Euclidean geometry of the highly-convoluted brain surface. Recent advances in machine learning have enabled the use of neural networks for non-Euclidean spaces. These facilitate the learning of surface data, yet pooling strategies often remain constrained to a single fixed-graph. This paper proposes a new learnable graph pooling method for processing multiple surface-valued data to output subject-based information. The proposed method innovates by learning an intrinsic aggregation of graph nodes based on graph spectral embedding. We illustrate the advantages of our approach with in-depth experiments on two large-scale benchmark datasets. The ablation study in the paper illustrates the impact of various factors affecting our learnable pooling method. The flexibility of the pooling strategy is evaluated on four different prediction tasks, namely, subject-sex classification, regression of cortical region sizes, classification of Alzheimer's disease stages, and brain age regression. Our experiments demonstrate the superiority of our learnable pooling approach compared to other pooling techniques for graph convolutional networks, with results improving the state-of-the-art in brain surface analysis.

We propose a novel pairwise distance measure between image keypoint sets, for the purpose of large-scale medical image indexing. Our measure generalizes the Jaccard index to account for soft set equivalence (SSE) between keypoint elements, via an adaptive kernel framework modeling uncertainty in keypoint appearance and geometry. A new kernel is proposed to quantify the variability of keypoint geometry in location and scale. Our distance measure may be estimated between${O}\,{(}{N}^{{\,{2}}}{)}$image pairs in${O}\,{(}{N}\,\text {log}{N}\,{)}$operations via keypoint indexing. Experiments report the first results for the task of predicting family relationships from medical images, using 1010 T1-weighted MRI brain volumes of 434 families including monozygotic and dizygotic twins, siblings and half-siblings sharing 100%-25% of their polymorphic genes. Soft set equivalence and the keypoint geometry kernel improve upon standard hard set equivalence (HSE) and appearance kernels alone in predicting family relationships. Monozygotic twin identification is near 100%, and three subjects with uncertain genotyping are automatically paired with their self-reported families, the first reported practical application of image-based family identification. Our distance measure can also be used to predict group categories, sex is predicted with an AUC = 0.97. Software is provided for efficient fine-grained curation of large, generic image datasets.

This article proposes to encode the distribution of features learned from a convolutional neural network (CNN) using a Gaussian mixture model (GMM). These parametric features, called GMM-CNN, are derived from chest computed tomography (CT) and X-ray scans of patients with coronavirus disease 2019 (COVID-19). We use the proposed GMM-CNN features as input to a robust classifier based on random forests (RFs) to differentiate between COVID-19 and other pneumonia cases. Our experiments assess the advantage of GMM-CNN features compared with standard CNN classification on test images. Using an RF classifier (80% samples for training; 20% samples for testing), GMM-CNN features encoded with two mixture components provided a significantly better performance than standard CNN classification ($p < 0.05$). Specifically, our method achieved an accuracy in the range of 96.00%–96.70% and an area under the receiver operator characteristic (ROC) curve in the range of 99.29%–99.45%, with the best performance obtained by combining GMM-CNN features from both CT and X-ray images. Our results suggest that the proposed GMM-CNN features could improve the prediction of COVID-19 in chest CT and X-ray scans.

The contrastive pre-training of a recognition model on a large dataset of unlabeled data often boosts the model’s performance on downstream tasks like image classification. However, in domains such as medical imaging, collecting unlabeled data can be challenging and expensive. In this work, we consider the task of medical image segmentation and adapt contrastive learning with meta-label annotations to scenarios where no additional unlabeled data is available. Meta-labels, such as the location of a 2D slice in a 3D MRI scan, often come for free during the acquisition process. We use these meta-labels to pre-train the image encoder, as well as in a semi-supervised learning step that leverages a reduced set of annotated data. A self-paced learning strategy exploiting the weak annotations is proposed to furtherhelp the learning process and discriminate useful labels from noise. Results on five medical image segmentation datasets show that our approach: i) highly boosts the performance of a model trained on a few scans, ii) outperforms previous contrastive and semi-supervised approaches, and iii) reaches close to the performance of a model trained on the full data.

) compared to 64.0% (p = 0.01) for the 3D CNN model output, 66.8% (p = 0.01) for standard radiomic features, 64.2% (p = 0.003) for CENT, and 57.6% (p = 0.3) for clinical variables. Our results suggest that the proposed GMM-CNN features used with a RF classifier can significantly improve the capacity to prognosticate PDAC patients prior to surgery via routinely-acquired imaging data.

This paper presents a client/server privacy-preserving network in the context of multicentric medical image analysis. Our approach is based on adversarial learning which encodes images to obfuscate the patient identity while preserving enough information for a target task. Our novel architecture is composed of three components: 1) an encoder network which removes identity-specific features from input medical images, 2) a discriminator network that attempts to identify the subject from the encoded images, 3) a medical image analysis network which analyzes the content of the encoded images (segmentation in our case). By simultaneously fooling the discriminator and optimizing the medical analysis network, the encoder learns to remove privacy-specific features while keeping those essentials for the target task. Our approach is illustrated on the problem of segmenting brain MRI from the large-scale Parkinson Progression Marker Initiative (PPMI) dataset. Using longitudinal data from PPMI, we show that the discriminator learns to heavily distort input images while allowing for highly accurate segmentation results. Our results also demonstrate that an encoder trained on the PPMI dataset can be used for segmenting other datasets, without the need for retraining. The code is made available at: https://github.com/bachkimn/Privacy-Net-An-Adversarial-Approach-forIdentity-Obfuscated-Segmentation-of-MedicalImages.

In recent years, approaches based on nonlocal self similarity and global structure regularization have led to significant improvements in image restoration. Nonlocal self similarity exploits the repetitiveness of small image patches as a powerful prior in the reconstruction process. Likewise, global structure regularization is based on the principle that the structure of objects in the image is represented by a relatively small portion of pixels. Enforcing this structural information to be sparse can thus reduce the occurrence of reconstruction artifacts. So far, most image restoration approaches have considered one of these two strategies, but not both. This paper presents a novel image restoration method that combines nonlocal self similarity and global structure sparsity in a single efficient model. Group of similar patches are reconstructed simultaneously, via an adaptive regularization technique based on the weighted nuclear norm. Moreover, global structure is preserved using an innovative strategy, which decomposes the image into a smooth component and a sparse residual, the latter regularized using l1 norm. An optimization technique, based on the Alternating Direction Method of Multipliers (ADMM) algorithm, is used to recover corrupted images efficiently. The performance of the proposed method is evaluated on two important image restoration tasks: image completion and super-resolution. Experimental results show our method to outperform state-of-the-art approaches for these tasks, for various types and levels of image corruption.

This paper presents a novel set of image texture features generalizing standard grey-level co-occurrence matrices (GLCM) to multimodal image data through joint intensity matrices (JIMs). These are used to predict the survival of glioblastoma multiforme (GBM) patients from multimodal MRI data. The scans of 73 GBM patients from the Cancer Imaging Archive are used in our study. Necrosis, active tumor, and edema/invasion subregions of GBM phenotypes are segmented using the coregistration of contrast-enhanced T1-weighted (CE-T1) images and its corresponding fluid-attenuated inversion recovery (FLAIR) images. Texture features are then computed from the JIM of these GBM subregions and a random forest model is employed to classify patients into short or long survival groups. Our survival analysis identified JIM features in necrotic (e.g., entropy and inverse-variance) and edema (e.g., entropy and contrast) subregions that are moderately correlated with survival time (i.e., Spearman rank correlation of 0.35). Moreover, nine features were found to be associated with GBM survival with a Hazard-ratio range of 0.38-2.1 and a significance level of p < 0.05 following Holm-Bonferroni correction. These features also led to the highest accuracy in a univariate analysis for predicting the survival group of patients, with AUC values in the range of 68-70%. Considering multiple features for this task, JIM features led to significantly higher AUC values than those based on standard GLCMs and gene expression. Furthermore, an AUC of 77.56% with p = 0.003 was achieved when combining JIM, GLCM, and gene expression features into a single radiogenomic signature. In summary, our study demonstrated the usefulness of modeling the joint intensity characteristics of CE-T1 and FLAIR images for predicting the prognosis of patients with GBM.

Recently, dense connections have attracted substantial attention in computer vision because they facilitate gradient flow and implicit deep supervision during training. Particularly, DenseNet that connects each layer to every other layer in a feed-forward fashion and has shown impressive performances in natural image classification tasks. We propose HyperDenseNet, a 3-D fully convolutional neural network that extends the definition of dense connectivity to multi-modal segmentation problems. Each imaging modality has a path, and dense connections occur not only between the pairs of layers within the same path but also between those across different paths. This contrasts with the existing multi-modal CNN approaches, in which modeling several modalities relies entirely on a single joint layer (or level of abstraction) for fusion, typically either at the input or at the output of the network. Therefore, the proposed network has total freedom to learn more complex combinations between the modalities, within and in-between all the levels of abstraction, which increases significantly the learning representation. We report extensive evaluations over two different and highly competitive multi-modal brain tissue segmentation challenges, iSEG 2017 and MRBrainS 2013, with the former focusing on six month infant data and the latter on adult images. HyperDenseNet yielded significant improvements over many state-of-the-art segmentation networks, ranking at the top on both benchmarks. We further provide a comprehensive experimental analysis of features re-use, which confirms the importance of hyper-dense connections in multi-modal representation learning. Our code is publicly available.

Accurate segmentation of infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) is an indispensable foundation for early studying of brain growth patterns and morphological changes in neurodevelopmental disorders. Nevertheless, in the isointense phase (approximately 6-9 months of age), due to inherent myelination and maturation process, WM and GM exhibit similar levels of intensity in both T1-weighted (T1w) and T2-weighted (T2w) MR images, making tissue segmentation very challenging. Despite many efforts were devoted to brain segmentation, only few studies have focused on the segmentation of 6-month infant brain images. With the idea of boosting methodological development in the community, iSeg-2017 challenge (http://iseg2017.web.unc.edu) provides a set of 6-month infant subjects with manual labels for training and testing the participating methods. Among the 21 automatic segmentation methods participating in iSeg-2017, we review the 8 top-ranked teams, in terms of Dice ratio, modified Hausdorff distance and average surface distance, and introduce their pipelines, implementations, as well as source codes. We further discuss limitations and possible future directions. We hope the dataset in iSeg-2017 and this review article could provide insights into methodological development for the community.

Total variation (TV) and non-local patch similarity have been used successfully to enhance the performance of compressive sensing (CS) approaches. However, such techniques can often remove important details in the image or introduce reconstruction artifacts. This paper presents a novel CS method, which uses an adaptive reweighted TV strategy to better preserve image edges. Our method also leverages the redundancy of non-local image patches through the use of weighted low rank regularization. An optimization strategy based on the ADMM algorithm is used to reconstruct images efficiently. Experimental results show our method to outperform state-of-the-art CS approaches, for various sampling ratios.

Word spotting in document images is a challenging problem, due to the large intra-class variability in handwritten shapes and the lack of labeled data. To tackle these challenges, this paper proposes an efficient multiscale representation for word images, which is learned in an unsupervised manner using the spherical k-means algorithm. A pooling function is applied in a spatial grid to obtain a fixed-length vector of features, robust to small shifts in the image. Scale variability in handwritten data is also considered by using patches of various sizes in the encoding process. Another important contribution of this work is to model the training-based word spotting task as a classification problem in the space of representation differences, thereby allowing the learned model to find matches for word classes that were not seen in training. The proposed system is evaluated on the well-known George Washington (GW) dataset. Experimental results show that our system outperforms state-of-the-art word spotting approaches in both training-free and training-based scenarios.

Structure and nonlocal patch similarity have been used successfully to enhance the performance of image restoration. However, these techniques can often remove textures and edges, or introduce artifacts. In this paper, we propose a novel image completion method that leverages the redundancy of nonlocal image patches via the low-rank regularization of similar patch groups. The textures and edges in these patches are preserved using an adaptive regularization technique based on the weighted nuclear norm. Furthermore, a new global structure regularization strategy, imposing ℓ1-norm sparsity on the image's high-frequency residual component, is presented to recover missing pixels while preserving structural information in the image. An efficient optimization technique, based on the Alternating Direction Method of Multipliers (ADMM) algorithm, is used to solve the proposed model. Experimental results show our method to outperform state-of-the-art image completion approaches, for various text-corrupted images and different ratios of missing pixels.

Various image priors, such as sparsity prior, non‐local self‐similarity prior and gradient histogram prior, have been widely used for noise removal, while preserving the image texture. However, the gradient histogram prior used for texture enhancement sometimes generates false textures in the smooth areas. In order to address these problems, the authors propose a robust algorithm combining gradient histogram with sparse representation to obtain good estimates of the sparse coding coefficients of the latent image and realising image denoising while preserving the texture. The proposed model is solved by having a balance between over‐enhancement and over‐smoothing of the texture in order to preserve the natural texture appearance. Experimental results demonstrate the efficiency and effectiveness of the proposed method.

This paper proposes a novel super-resolution method that exploits the sparse representation and non-local similarity of patches for the effective reconstruction of images. Highresolution images are reconstructed from low resolution observations with an efficient technique based on the alternating direction method of multipliers (ADMM). A robust iterative back-projection approach is used in a post-processing step to remove residual noise and artifacts in the reconstructed image. Experiments on benchmark medical images illustrate the advantage of our method, in terms of PSNR and SSIM, compared to state of the art approaches.

This paper proposes to use texture features extracted from multispectral microscopic images to detect histopathological abnormalities related to colorectal cancer (CRC): stroma (ST), benign hyperplasia (BH), intraepithelial neoplasia (IN) and carcinoma (Ca). Texture features, based on gray-level co-occurrence matrices (GLCM) and discrete wavelets (DW), are obtained from colon biopsy images, captured using 16 different bands of the visible spectrum. A random forest classifier is used to evaluate the usefulness of these texture features, for each spectral band, on the task of discriminating between the four types of abnormal tissue. Preliminary results on the data of 39 CRC patients show that such features, in particular those based on GLCM and Symlet wavelets, can accurately predict the type of CRC tissue (94% accuracy, 88% sensibility and 100% specificity for Symlet features in the 16thspectral band). These results also reveal important differences in the textural information captured in each band, which could be used to develop more efficient procedures for the diagnosis of CRC.

In the context of motion estimation (ME) for video coding, the rate-constrained successive elimination algorithm (RC-SEA) safely eliminates candidate motion vectors while preserving the optimal candidate chosen by the block matching algorithm (BMA). This paper describes a technique for reusing ME information from rectangular to square prediction units in order to reduce the search area without altering the optimal candidate chosen by the BMA. Our experiments show that, on average, when this optimization is combined with the RCSEA in the HEVC HM encoder reference software, the number of sum of the absolute differences (SAD) operations drops by 94.9%, resulting in a speedup of 6.13x in full search mode. Although identical coding decisions cannot be guaranteed when multiple optimal solutions exist, the average impact on BD-PSNR is 0.0002 dB.

Compressed sensing is a powerful approach to reconstruct high-quality images using a small number of samples. This paper presents a novel compressed sensing method that uses a probabilistic atlas to impose spatial constraints on the reconstruction of brain magnetic resonance imaging (MRI) data. A weighted total variation (TV) model is proposed to characterize the spatial distribution of gradients in the brain, and incorporate this information in the reconstruction process. Experiments on T1-weighted MR images from the ABIDE dataset show our proposed method to outperform the standard uniform TV model, as well as state-of-the-art approaches, for low sampling rates and high noise levels.

Sparse multivariate regression techniques like Lasso and Elastic Net are among the most popular approaches for the identification of biomarkers related to brain diseases like Alzheimer's. Because they use L1norm to enforce sparsity, these approaches are often sensitive to differences in voxel intensities within the same scan or across subjects. Also, when few samples are available, such approaches can select voxels that are only correlated by chance, leading to disconnected features that do not correspond to any significant brain structure. To address these challenges, we propose a novel sparse regression method that uses the L0norm for sparse regularization, and imposes spatial consistency constraints on the selected features without requiring an atlas of pre-defined regions. This method uses an efficient optimization strategy based on the Alternating Direction Method of Multipliers (ADMM), that can scale to large data matrices. The performance of the proposed method is evaluated using synthetic data and 3429 T1-weighted (MP-RAGE) images from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Experimental results show our method to outperform Lasso and Elastic Net regression in the recovery of spatially consistent features corresponding to known neuroimaging biomarkers.

The sparse representation models for image super-resolution have shown great potential in various imaging and vision tasks. However, most of them are challenged by the accuracy issue especially when images are significantly down-sampled. In this paper, we aim to improve the performance of sparse representation. We propose to incorporate a low rank approach into image non-local sparse representation model. To the best of our knowledge, this is the first work to integrate low rank approaches into non-local spare representation for image interpolation. The proposed method can obtain good estimation of sparse coefficients of original images. Experimental results show the effectiveness of our proposed method compared with the state-of-the-art.

Total variation (TV) based sparsity and non local self-similarity have been shown to be powerful tools for the reconstruction of magnetic resonance (MR) images. However, due to the uniform regularization of gradient sparsity, standard TV approaches often over-smooth edges in the image, resulting in the loss of important details. This paper presents a novel compressed sensing method for the reconstruction of MRI data, which uses a regularization strategy based on re-weighted TV to preserve image edges. This method also leverages the redundancy of non local image patches through the use of a sparse regression model. An efficient strategy based on the Alternating Direction Method of Multipliers (ADMM) algorithm is used to recover images with the proposed model. Experimental results on a simulated phantom and real brain MR data show our method to outperform state-of-the-art compressed sensing approaches, by better preserving edges and removing artifacts in the image.

This paper presents a new segmentation method that combines random walks with superpixels. In this method, the coarseness of the segmentation is controlled using a single parameter. By using superpixels, the method can recompute the segmentation efficiently, making the parameter tuning process interactive. Moreover, an efficient strategy is proposed to adjust dynamically the parameters to the image's content, making our method more robust than existing approaches. Experiments performed on the Berkeley BSD300 segmentation database show our interactive method to outperform state-of-the-art approaches for this task.

We present a semi-supervised segmentation method that uses a dictionary of multi-class foreground histograms to enhance the segmentation in the presence of incorrect or missing labels. Instead of requiring a target histogram, or a set of images with the same foreground, this method uses sparse coding to find the most relevant histogram for the foreground. An efficient strategy based on the ADMM algorithm is proposed to avoid the problems of non-submodularity and non-linearity, normally related to histogram-based segmentation. Experiments on the segmentation of natural images with incomplete or incorrect labels show our method to be more robust and accurate than other approaches for this task.

Coronary arteries (CA) segmentation from an angiographic sequence is essential to guide the cardiologists during percutaneous interventions for the treatment and diagnosis of pathologies. Segmentation of the CA from X-ray angiograms is a very challenging problem due to the changes in contrast in the sequence in addition to the CA's complex topology. In this paper, we propose a hierarchical segmentation method that extends the Vessel Walker model using a temporal prior and multiscale information to extract CA with a higher level of accuracy. In this method, the vessel located in frame Itat time t is extracted by utilizing the segmentation result at frame It−1together with Histogram of Oriented Gradient (HOG) features and a shape matching technique. Our experiments conducted on five paediatric angiograms have shown promising qualitative and quantitative results with a mean Dice coefficient of 64% and 53% Recall and 85% in Precision.

This paper proposes a solution for the problem of unnecessary cost function evaluations, found when combining the successive elimination algorithm with a spiral scan search ordering. Our experiments show that the implementation of such a combination inside the HEVC reference software leads to unnecessary cost function evaluations. On the tested video sequences, an average of 3.46% unnecessary cost function evaluations was measured. Considering only small block sizes (e.g., 4×8 and 8×4), this average rises to 8.06%. To solve this problem, we propose an adaptive scan ordering of block matching candidates within the search area. When used with our early termination threshold, the proposed approach will only evaluate necessary cost functions, without impacting rate-distortion.

Traditional deblurring algorithms are often focused on natural-scaled images, which are not adapted for document texts and images without having some negative impacts on the accuracy of the OCR and the visual quality. In this paper, we propose a gradient histogram preservation method. An effective optimization method was developed and achieves satisfying results for kernel estimation. By combining the gradient histogram preservation prior with conventional image deblurring methods, it significantly improves the simulations and experimental results on document images and a high SSIM is achieved with the proposed method.

In this paper, we propose a new class of search ordering algorithms to reduce the computational cost of motion estimation in video coding. We show that conventional search orderings, such as spiral search, can weaken the filtering criterion of rate-constrained successive elimination algorithms. Based on this new insight, we derive a new search ordering that takes into account the impact of the rate constraint. Our simulation results demonstrate that, on average, the amount of SAD operations required to encode the tested sequences, is reduced by 2.86%, when compared to the H.264 JM reference software's implementation of spiral search. For sequences with unpredictable motion, this reduction is greater than 5% and can exceed 10% when smaller block partitions are evaluated.

Image segmentation is a challenging task that has several applications in domains like medical imaging and surveillance. Among the various approaches proposed for this task, unsupervised methods have the advantage of being able to segment images without any assistance from the user. However, such methods often suffer from long runtimes and tend to be sensitive to the choice of parameters. Because of these problems, users will often prefer semi-supervised methods, which provide a more controllable output in the same amount of time. This paper proposes a new unsupervised approach, based on random walks, which maps each pixel to the most probable label in a local neighborhood. To make this approach more robust to the choice and learning of the parameters, we propose an efficient computational technique, in which the parameters and the segmentation probabilities are recomputed alternatively. We also describe a refinement strategy that improves the speed and accuracy of the segmentation by applying random walks at different scales. We evaluate the usefulness of our approach on the segmentation of natural images from the Berkeley segmentation database (BSD300). Results show our approach to have an accuracy comparable to state-of-the-art segmentation methods, while being much faster than these methods.