Recent advancements in vision-language models (VLMs), such as CLIP, have demonstrated substantial success in self-supervised representation learning for vision tasks. However, effectively adapting VLMs to downstream applications remains challenging, as their accuracy often depends on time-intensive and expertise-demanding prompt engineering, while full model fine-tuning is costly. This is particularly true for biomedical images, which, unlike natural images, typically suffer from limited annotated datasets, unintuitive image contrasts, and nuanced visual features. Recent prompt learning techniques, such as Context Optimization (CoOp) intend to tackle these issues, but still fall short in generalizability. Meanwhile, explorations in prompt learning for biomedical image analysis are still highly limited. In this work, we propose BiomedCoOp, a novel prompt learning framework that enables efficient adaptation of BiomedCLIP for accurate and highly generalizable few-shot biomedical image classification. Our approach achieves effective prompt context learning by leveraging semantic consistency with average prompt ensembles from Large Language Models (LLMs) and knowledge distillation with a statistics-based prompt selection strategy. We conducted comprehensive validation of our proposed framework on 11 medical datasets across 9 modalities and 10 organs against existing state-of-the-art methods, demonstrating significant improvements in both accuracy and generalizability. The code is publicly available at https://github.com/HealthX-Lab/BiomedCoOp.

With the widespread interest and uptake of super-resolution ultrasound (SRUS) through localization and tracking of microbubbles, also known as ultrasound localization microscopy (ULM), many localization and tracking algorithms have been developed. ULM can image many centimeters into tissue in-vivo and track microvascular flow non-invasively with sub-diffraction resolution. In a significant community effort, we organized a challenge, Ultrasound Localization and TRacking Algorithms for Super-Resolution (ULTRA-SR). The aims of this paper are threefold: to describe the challenge organization, data generation, and winning algorithms; to present the metrics and methods for evaluating challenge entrants; and to report results and findings of the evaluation. Realistic ultrasound datasets containing microvascular flow for different clinical ultrasound frequencies were simulated, using vascular flow physics, acoustic field simulation and nonlinear bubble dynamics simulation. Based on these datasets, 38 submissions from 24 research groups were evaluated against ground truth using an evaluation framework with six metrics, three for localization and three for tracking. In-vivo mouse brain and human lymph node data were also provided, and performance assessed by an expert panel. Winning algorithms are described and discussed. The publicly available data with ground truth and the defined metrics for both localization and tracking present a valuable resource for researchers to benchmark algorithms and software, identify optimized methods/software for their data, and provide insight into the current limits of the field. In conclusion, Ultra-SR challenge has provided benchmarking data and tools as well as direct comparison and insights for a number of the state-of-the art localization and tracking algorithms.

One of the primary sources of suboptimal image quality in ultrasound imaging is phase aberration. It is caused by spatial changes in sound speed over a heterogeneous medium, which disturbs the transmitted waves and prevents coherent summation of echo signals. Obtaining non-aberrated ground truths in real-world scenarios can be extremely challenging, if not impossible. This challenge hinders the performance of deep learning-based techniques due to the domain shift between simulated and experimental data. Here, for the first time, we propose a deep learning-based method that does not require ground truth to correct the phase aberration problem and, as such, can be directly trained on real data. We train a network wherein both the input and target output are randomly aberrated radio frequency (RF) data. Moreover, we demonstrate that a conventional loss function such as mean square error is inadequate for training such a network to achieve optimal performance. Instead, we propose an adaptive mixed loss function that employs both B-mode and RF data, resulting in more efficient convergence and enhanced performance. Finally, we publicly release our dataset, comprising over 180,000 aberrated single plane-wave images (RF data), wherein phase aberrations are modeled as near-field phase screens. Although not utilized in the proposed method, each aberrated image is paired with its corresponding aberration profile and the non-aberrated version, aiming to mitigate the data scarcity problem in developing deep learning-based techniques for phase aberration correction. Source code and trained model are also available along with the dataset at https://code.sonography.ai/main-aaa.

Tracking the displacement between the pre- and post-deformed radio-frequency (RF) frames is a pivotal step of ultrasound elastography, which depicts tissue mechanical properties to identify pathologies. Due to ultrasound's poor ability to capture information pertaining to the lateral direction, the existing displacement estimation techniques fail to generate an accurate lateral displacement or strain map. The attempts made in the literature to mitigate this well-known issue suffer from one of the following limitations: 1) Sampling size is substantially increased, rendering the method computationally and memory expensive. 2) The lateral displacement estimation entirely depends on the axial one, ignoring data fidelity and creating large errors. This paper proposes exploiting the effective Poisson's ratio (EPR)-based mechanical correspondence between the axial and lateral strains along with the RF data fidelity and displacement continuity to improve the lateral displacement and strain estimation accuracies. We call our techniques MechSOUL (Mechanically-constrained Second-Order Ultrasound eLastography) and L1 -MechSOUL ( L1 -norm-based MechSOUL), which optimize L2 - and L1 -norm-based penalty functions, respectively. Extensive validation experiments with simulated, phantom, and in vivo datasets demonstrate that MechSOUL and L1 -MechSOUL's lateral strain and EPR estimation abilities are substantially superior to those of the recently-published elastography techniques. We have published the MATLAB codes of MechSOUL and L1 -MechSOUL at https://code.sonography.ai.

Convolutional Neural Networks (CNN) have shown promising results for displacement estimation in UltraSound Elastography (USE). Many modifications have been proposed to improve the displacement estimation of CNNs for USE in the axial direction. However, the lateral strain, which is essential in several downstream tasks such as the inverse problem of elasticity imaging, remains a challenge. The lateral strain estimation is complicated since the motion and the sampling frequency in this direction are substantially lower than the axial one, and a lack of carrier signal in this direction. In computer vision applications, the axial and the lateral motions are independent. In contrast, the tissue motion pattern in USE is governed by laws of physics which link the axial and lateral displacements. In this paper, inspired by Hooke's law, we, first propose Physically Inspired ConsTraint for Unsupervised Regularized Elastography (PICTURE), where we impose a constraint on the Effective Poisson's ratio (EPR) to improve the lateral strain estimation. In the next step, we propose self-supervised PICTURE (sPICTURE) to further enhance the strain image estimation. Extensive experiments on simulation, experimental phantom and in vivo data demonstrate that the proposed methods estimate accurate axial and lateral strain maps.

In this work, a novel technique for real-time clutter rejection in ultrasound Color Flow Imaging (CFI) is proposed. Suppressing undesired clutter signal is important because clutter prohibits an unambiguous view of the vascular network. Although conventional eigen-based filters are potentially efficient in suppressing clutter signal, their performance is highly dependent on proper selection of a clutter to blood boundary which is done manually. Herein, we resolve this limitation by formulating the clutter suppression problem as a foreground-background separation problem to extract the moving blood component. To that end, we adapt the fast Robust Matrix Completion (fRMC) algorithm, and utilize the in-face extended Frank-Wolfe method to minimize the rank of the matrix of ultrasound frames. Our method automates the clutter suppression process, which is critical for clinical use. We name the method RAPID (Robust mAtrix decomPosition for suppressIng clutter in ultrasounD) since the automation step can substantially streamline clutter suppression. The technique is validated with simulation, flow phantom and two sets of in-vivo data. RAPID code as well as most of the data in this paper can be downloaded from RAPID.sonography.ai.

In brain tumor surgery, the quality and safety of the procedure can be impacted by intra-operative tissue deformation, called brain shift. Brain shift can move the surgical targets and other vital structures such as blood vessels, thus invalidating the pre-surgical plan. Intra-operative ultrasound (iUS) is a convenient and cost-effective imaging tool to track brain shift and tumor resection. Accurate image registration techniques that update pre-surgical MRI based on iUS are crucial but challenging. The MICCAI Challenge 2018 for Correction of Brain shift with Intra-Operative UltraSound (CuRIOUS2018) provided a public platform to benchmark MRI-iUS registration algorithms on newly released clinical datasets. In this work, we present the data, setup, evaluation, and results of CuRIOUS 2018, which received 6 fully automated algorithms from leading academic and industrial research groups. All algorithms were first trained with the public RESECT database, and then ranked based on a test dataset of 10 additional cases with identical data curation and annotation protocols as the RESECT database. The article compares the results of all participating teams and discusses the insights gained from the challenge, as well as future work.

A major challenge of free-hand palpation ultrasound elastography (USE) is estimating the displacement of RF samples between pre- and post-compressed RF data. The problem of displacement estimation is ill-posed since the displacement of one sample by itself cannot be uniquely calculated. To resolve this problem, two categories of methods have emerged. The first category assumes that the displacement of samples within a small window surrounding the reference sample is constant. The second class imposes smoothness regularization and optimizes an energy function. Herein, we propose a novel method that combines both approaches, and as such, is more robust to noise. The second contribution of this work is the introduction of the L1 norm as the regularization term in our cost function, which is often referred to as the total variation (TV) regularization. Compared to previous work that used the L2 norm regularization, optimization of the new cost function is more challenging. However, the advantages of using the L1 norm are twofold. First, it leads to substantial improvement in the sharpness of displacement estimates. Second, to optimize the cost function with the L1 norm regularization, we use an iterative method that further increases the robustness. We name our proposed method tOtal Variation Regularization and WINDow-based time delay estimation (OVERWIND) and show that it is robust to signal decorrelation and generates sharp displacement and strain maps for simulated, experimental phantom and in-vivo data. In particular, OVERWIND improves strain contrast-to-noise ratio (CNR) by 27.26%, 144.05%, and 49.90% on average in simulation, phantom, and in-vivo data, respectively, compared to our recent Global Ultrasound Elastography (GLUE) method.

The resolution of ultrasound (US) images is limited by physical constraints and hardware restrictions, such as the frequency, width and focal zone of the US beam. Different interpolation methods are often used to increase the sampling rate of ultrasound images. However, interpolation methods generally introduce blur in images. Herein, we present a super resolution (SR) algorithm for reconstruction of the B-mode images using the information from the envelope of radio frequency (RF) data. Our method is based on utilizing repetitive data in the nonlocal neighborhood of samples. The performance of the proposed approach is determined both qualitatively and quantitatively using phantom and in-vivo data.

Image guidance has become the standard of care for patient positioning in radiotherapy, where image registration is often a critical step to help manage patient motion. However, in practice, verification of registration quality is often adversely affected by difficulty in manual inspection of 3-D images and time constraint, thus affecting the therapeutic outcome. Therefore, we proposed to employ both bootstrapping and the supervised learning methods of linear discriminant analysis and random forest to help robustly assess registration quality in ultrasound-guided radiotherapy. We validated both approaches using phantom and real clinical ultrasound images, and showed that both performed well for the task. While learning-based techniques offer better accuracy and shorter evaluation time, bootstrapping requires no prior training and has a higher sensitivity.

Ultrasound elastography involves imaging tissue while it undergoes deformation and inferring its mechanical properties from the deformation pattern. The initial deformation in the tissue is typically induced through an external mechanical force, for example, by exerting a slight pressure using an ultrasound probe or by applying an acoustic radiation force (ARF) against the tissue. The ARF excites the tissue locally, which leads to the propagation of a shear-wave. The goal of the shear-wave elastography is to estimate the speed of the shear-wave that is explicitly related to the elasticity of tissue. We formulate tissue displacement estimation as an optimization problem and propose a computationally efficient approach to estimate the displacement field. A novel algorithm based on the minimization of a regularized cost function using higher-order analytical minimization (HAM) coupled with the second-order Taylor series approximation is proposed. Our algorithm first computes an integer displacement field based on dynamic programming (DP) that provides the global optima, which is then refined iteratively to obtain the subpixel displacement estimate. We test the proposed algorithm on real experimental data obtained from a tissue-mimicking phantom and illustrate the superiority of our approach over some commonly used elastography techniques using signal to noise ratio (SNR) comparisons.

Most strain imaging techniques follow a pipeline strategy: in the first step, tissue displacement is estimated from radio-frequency (RF) frames, and in the second step, a spatial derivative operation is applied. There are two main issues that arise from this framework. First, the gradient operation amplifies noise, and therefore, smoothing techniques have to be adopted. Second, strain estimation does not exploit the original RF data. It rather relies solely on the noisy displacement field. In this paper, a novel technique is proposed that utilizes both the displacement field and the RF frames to accurately obtain the strain estimates. The normalized cross correlation (NCC) metric between two corresponding windows around the samples of the pre- and post-compressed images is employed to generate a dissimilarity measurement. The derivative of NCC with respect to the strain is analytically derived using the chain rule. This allows an efficient minimization of the dissimilarity metric with respect to the strain using the gradient descent optimization technique. The effectiveness of the proposed method is investigated through simulation data, phantom experiments, and in vivo patient data. The experimental results show that exploiting the information in RF data significantly improves the strain estimates.

Brain tissue deforms significantly after opening the dura and during tumor resection, invalidating preoperative imaging data. Ultrasound is a popular imaging modality for providing the neurosurgeon with real-time updated images of brain tissue. Interpretation of postresection ultrasound images is difficult due to large brain shift and tissue resection. Furthermore, several factors degrade the quality of postresection ultrasound images such as the strong reflection of waves at the interface of saline water and brain tissue in resection cavities, air bubbles, and the application of blood-clotting agents around the edges of resection. Image registration allows the comparison of postresection ultrasound images with higher quality preresection images, assists in interpretation of postresection images and may help identify residual tumor, and, as such, is of significant clinical importance. In this paper, we propose a nonrigid symmetric registration (NSR) framework for accurate alignment of pre- and postresection volumetric ultrasound images in near real time. We first formulate registration as minimization of a regularized cost function and analytically derive its derivative to efficiently optimize the cost function. An outlier detection algorithm is proposed and utilized in this framework to identify noncorresponding regions (outliers) and therefore improve the robustness and accuracy of registration. We use an efficient second-order minimization method for fast and robust optimization. Furthermore, we exploit a symmetric and inverse-consistent method to generate realistic deformation fields. The results show that NSR significantly improves the quality of the alignment between pre- and postresection ultrasound images.

We propose a conditional random field (CRF) based classifier for segmentation of small enhanced pathologies. Specifically, we develop a temporal hierarchical adaptive texture CRF (THAT-CRF) and apply it to the challenging problem of gad enhancing lesion segmentation in brain MRI of patients with multiple sclerosis. In this context, the presence of many nonlesion enhancements (such as blood vessels) renders the problem more difficult. In addition to voxel-wise features, the framework exploits multiple higher order textures to discriminate the true lesional enhancements from the pool of other enhancements. Since lesional enhancements show more variation over time as compared to the nonlesional ones, we incorporate temporal texture analysis in order to study the textures of enhanced candidates over time. The parameters of the THAT-CRF model are learned based on 2380 scans from a multi-center clinical trial. The effect of different components of the model is extensively evaluated on 120 scans from a separate multi-center clinical trial. The incorporation of the temporal textures results in a general decrease of the false discovery rate. Specifically, THAT-CRF achieves overall sensitivity of 95% along with false discovery rate of 20% and average false positive count of 0.5 lesions per scan. The sensitivity of the temporal method to the trained time interval is further investigated on five different intervals of 69 patients. Moreover, superior performance is achieved by the reviewed labelings of our model compared to the fully manual labeling when applied to the context of separating different treatment arms in a real clinical trial.

In this work, we present a novel algorithm for registration of 3-D volumetric ultrasound (US) and MR using Robust PaTch-based cOrrelation Ratio (RaPTOR). RaPTOR computes local correlation ratio (CR) values on small patches and adds the CR values to form a global cost function. It is therefore invariant to large amounts of spatial intensity inhomogeneity. We also propose a novel outlier suppression technique based on the orientations of the RaPTOR gradients. Our deformation is modeled with free-form cubic B-splines. We analytically derive the derivatives of RaPTOR with respect to the transformation, i.e., the displacement of the B-spline nodes, and optimize RaPTOR using a stochastic gradient descent approach. RaPTOR is validated on MR and tracked US images of neurosurgery. Deformable registration of the US and MR images acquired, respectively, preoperation and postresection is of significant clinical significance, but challenging due to, among others, the large amount of missing correspondences between the two images. This work is also novel in that it performs automatic registration of this challenging dataset. To validate the results, we manually locate corresponding anatomical landmarks in the US and MR images of tumor resection in brain surgery. Compared to rigid registration based on the tracking system alone, RaPTOR reduces the mean initial mTRE over 13 patients from 5.9 to 2.9 mm, and the maximum initial TRE from 17.0 to 5.9 mm. Each volumetric registration using RaPTOR takes about 30 sec on a single CPU core. An important challenge in the field of medical image analysis is the shortage of publicly available dataset, which can both facilitate the advancement of new algorithms to clinical settings and provide a benchmark for comparison. To address this problem, we will make our manually located landmarks available online.

Mutual information (MI) quantifies the information that is shared between two random variables and has been widely used as a similarity metric for multi-modal and uni-modal image registration. A drawback of MI is that it only takes into account the intensity values of corresponding pixels and not of neighborhoods. Therefore, it treats images as "bag of words" and the contextual information is lost. In this work, we present Contextual Conditioned Mutual Information (CoCoMI), which conditions MI estimation on similar structures. Our rationale is that it is more likely for similar structures to undergo similar intensity transformations. The contextual analysis is performed on one of the images offline. Therefore, CoCoMI does not significantly change the registration time. We use CoCoMI as the similarity measure in a regularized cost function with a B-spline deformation field and efficiently optimize the cost function using a stochastic gradient descent method. We show that compared to the state of the art local MI based similarity metrics, CoCoMI does not distort images to enforce erroneous identical intensity transformations for different image structures. We further present the results on nonrigid registration of ultrasound (US) and magnetic resonance (MR) patient data from image-guided neurosurgery trials performed in our institute and publicly available in the BITE dataset. We show that CoCoMI performs significantly better than the state of the art similarity metrics in US to MR registration. It reduces the average mTRE over 13 patients from 4.12 mm to 2.35 mm, and the maximum mTRE from 9.38 mm to 3.22 mm.

This paper introduces two real-time elastography techniques based on analytic minimization (AM) of regularized cost functions. The first method (1D AM) produces axial strain and integer lateral displacement, while the second method (2D AM) produces both axial and lateral strains. The cost functions incorporate similarity of radio-frequency (RF) data intensity and displacement continuity, making both AM methods robust to small decorrelations present throughout the image. We also exploit techniques from robust statistics to make the methods resistant to large local decorrelations. We further introduce Kalman filtering for calculating the strain field from the displacement field given by the AM methods. Simulation and phantom experiments show that both methods generate strain images with high SNR, CNR and resolution. Both methods work for strains as high as 10% and run in real-time. We also present in vivo patient trials of ablation monitoring. An implementation of the 2D AM method as well as phantom and clinical RF-data can be downloaded.

This paper introduces a 2-D strain imaging technique based on minimizing a cost function using dynamic programming (DP). The cost function incorporates similarity of echo amplitudes and displacement continuity. Since tissue deformations are smooth, the incorporation of the smoothness into the cost function results in reduced decorrelation noise. As a result, the method generates high-quality strain images of freehand palpation elastography with up to 10% compression, showing that the method is more robust to signal decorrelation (caused by scatterer motion in high axial compression and nonaxial motions of the probe) in comparison to the standard correlation techniques. The method operates in less than 1 s and is thus also potentially suitable for real time elastography.