• A new method, UNISELF, is proposed to improve multiple sclerosis lesion segmentation. • UNISELF uses self-ensembled lesion fusion to improve accuracy and generalization. • UNISELF uses test-time instance normalization to address latent feature distribution shift. • UNISELF is among the top methods in the ISBI lesion segmentation challenge. • UNISELF outperforms other benchmarks on various out-of-domain test datasets. Automated segmentation of multiple sclerosis (MS) lesions using multicontrast magnetic resonance (MR) images improves efficiency and reproducibility compared to manual delineation, with deep learning (DL) methods achieving state-of-the-art performance. However, these DL-based methods have yet to simultaneously optimize in-domain accuracy and out-of-domain generalization when trained on a single source with limited data, or their performance has been unsatisfactory. To fill this gap, we propose a method called UNISELF, which achieves high accuracy within a single training domain while demonstrating strong generalizability across multiple out-of-domain test datasets. UNISELF employs a novel test-time self-ensembled lesion fusion to improve segmentation accuracy, and leverages test-time instance normalization (TTIN) of latent features to address domain shifts and missing input contrasts. Trained on the ISBI 2015 longitudinal MS segmentation challenge training dataset, UNISELF ranks among the best-performing methods on the challenge test dataset. Additionally, UNISELF outperforms all benchmark methods trained on the same ISBI training data across diverse out-of-domain test datasets with domain shifts and missing contrasts, including the public MICCAI 2016 and UMCL datasets, as well as a private multisite dataset. These test datasets exhibit domain shifts and/or missing contrasts caused by variations in acquisition protocols, scanner types, and imaging artifacts arising from imperfect acquisition. Our code is available at https://github.com/Jinwei1209/UNISELF .

Tagged magnetic resonance imaging (tMRI) is a valuable tool for visualizing and quantifying tissue deformation in vivo. Its use is often hampered, however, by tag fading, long computation times, and the challenge of ensuring diffeomorphic, incompressible motion fields. In this paper, we describe a novel integration of the harmonic phase (HARP) approach to tMRI analysis with an unsupervised deep learning-based registration framework to estimate 2D and 3D motion fields that are diffeomorphic and nearly incompressible. The resulting method, called deep sinusoidally transformed HARP, or DSHARP, enables end-to-end network training by implementing a transformation of the harmonic phase to remove phase-wrapping discontinuities. It produces diffeomorphic motion by estimating a stationary velocity field from which motion is computed using the scaling and squaring technique. Finally, it encourages incompressibility using a novel Jacobian determinant loss term during network training. We evaluated DSHARP on 2D and 3D phantom data with simulated incompressible motions, real 3D human tongue data acquired during speech from both healthy and glossectomy subjects, and cardiac tagged MRI from the public STACOM 2011 benchmark. Our approach outperforms HARP, SinMod, SyN, PVIRA, VoxelMorph, and DeepTag in tracking accuracy, computation speed, and preservation of incompressibility.

Abstract Producing spatial transformations that are diffeomorphic is a key goal in deformable image registration. As a diffeomorphic transformation should have positive Jacobian determinant $$\vert J\vert $$ | J | everywhere, the number of pixels (2D) or voxels (3D) with $$\vert J\vert <0$$ | J | < 0 has been used to test for diffeomorphism and also to measure the irregularity of the transformation. For digital transformations, $$\vert J\vert $$ | J | is commonly approximated using a central difference, but this strategy can yield positive $$\vert J\vert $$ | J | ’s for transformations that are clearly not diffeomorphic—even at the pixel or voxel resolution level. To show this, we first investigate the geometric meaning of different finite difference approximations of $$\vert J\vert $$ | J | . We show that to determine if a deformation is diffeomorphic for digital images, the use of any individual finite difference approximation of $$\vert J\vert $$ | J | is insufficient. We further demonstrate that for a 2D transformation, four unique finite difference approximations of $$\vert J\vert $$ | J | ’s must be positive to ensure that the entire domain is invertible and free of folding at the pixel level. For a 3D transformation, ten unique finite differences approximations of $$\vert J\vert $$ | J | ’s are required to be positive. Our proposed digital diffeomorphism criteria solves several errors inherent in the central difference approximation of $$\vert J\vert $$ | J | and accurately detects non-diffeomorphic digital transformations. The source code of this work is available at https://github.com/yihao6/digital_diffeomorphism .

In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.

Optical coherence tomography angiography (OCTA) is an imaging modality that can be used for analyzing retinal vasculature. Quantitative assessment of en face OCTA images requires accurate segmentation of the capillaries. Using deep learning approaches for this task faces two major challenges. First, acquiring sufficient manual delineations for training can take hundreds of hours. Second, OCTA images suffer from numerous contrast-related artifacts that are currently inherent to the modality and vary dramatically across scanners. We propose to solve both problems by learning a disentanglement of an anatomy component and a local contrast component from paired OCTA scans. With the contrast removed from the anatomy component, a deep learning model that takes the anatomy component as input can learn to segment vessels with a limited portion of the training images being manually labeled. Our method demonstrates state-of-the-art performance for OCTA vessel segmentation.