Over the past five years, artificial intelligence (AI) has introduced new models and methods for addressing the challenges associated with the broader adoption of AI models and systems in medicine. This paper reviews recent advances in AI for medical image and video analysis, outlines emerging paradigms, highlights pathways for successful clinical translation, and provides recommendations for future work. Hybrid Convolutional Neural Network (CNN) Transformer architectures now deliver state-of-the-art results in segmentation, classification, reconstruction, synthesis, and registration. Foundation and generative AI models enable the use of transfer learning to smaller datasets with limited ground truth. Federated learning supports privacy-preserving collaboration across institutions. Explainable and trustworthy AI approaches have become essential to foster clinician trust, ensure regulatory compliance, and facilitate ethical deployment. Together, these developments pave the way for integrating AI into radiology, pathology, and wider healthcare workflows.

Lung cancer is by far the leading cause of cancer death in the US. Recent studies have demonstrated the effectiveness of screening using low dose CT (LDCT) in reducing lung cancer related mortality. While lung nodules are detected with a high rate of sensitivity, this exam has a low specificity rate and it is still difficult to separate benign and malignant lesions. The ISBI 2018 Lung Nodule Malignancy Prediction Challenge, developed by a team from the Quantitative Imaging Network of the National Cancer Institute, was focused on the prediction of lung nodule malignancy from two sequential LDCT screening exams using automated (non-manual) algorithms. We curated a cohort of 100 subjects who participated in the National Lung Screening Trial and had established pathological diagnoses. Data from 30 subjects were randomly selected for training and the remaining was used for testing. Participants were evaluated based on the area under the receiver operating characteristic curve (AUC) of nodule-wise malignancy scores generated by their algorithms on the test set. The challenge had 17 participants, with 11 teams submitting reports with method description, mandated by the challenge rules. Participants used quantitative methods, resulting in a reporting test AUC ranging from 0.698 to 0.913. The top five contestants used deep learning approaches, reporting an AUC between 0.87 - 0.91. The team's predictor did not achieve significant differences from each other nor from a volume change estimate (p =.05 with Bonferroni-Holm's correction).

The papers in this special section focuses on artificial intelligent-driven informatics, sensing, imaging and big data analytics in dealing with the COVID-19 pandemic.

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

SCoTS captures a sparse representation of shapes in an input image through a linear span of previously delineated shapes in a training repository. The model updates shape prior over level set iterations and captures variabilities in shapes by a sparse combination of the training data. The level set evolution is therefore driven by a data term as well as a term capturing valid prior shapes. During evolution, the shape prior influence is adjusted based on shape reconstruction, with the assigned weight determined from the degree of sparsity of the representation. For the problem of lung nodule segmentation in X-ray CT, SCoTS offers a unified framework, capable of segmenting nodules of all types. Experimental validations are demonstrated on 542 3-D lung nodule images from the LIDC-IDRI database. Despite its generality, SCoTS is competitive with domain specific state of the art methods for lung nodule segmentation.

Magnetic resonance imaging (MRI) is a highly advanced and sophisticated imaging modality for cardiac motion tracking and analysis, capable of providing 3D analysis of global and regional cardiac function with great accuracy and reproducibility. In the past few years, numerous efforts have been devoted to cardiac motion recovery and deformation analysis from MR image sequences. Many approaches have been proposed for tracking cardiac motion and for computing deformation parameters and mechanical properties of the heart from a variety of cardiac MR imaging techniques. In this paper, an updated and critical review of cardiac motion tracking methods including major references and those proposed in the past ten years is provided. The MR imaging and analysis techniques surveyed are based on cine MRI, tagged MRI, phase contrast MRI, DENSE, and SENC. This paper can serve as a tutorial for new researchers entering the field.

We present research in which both left and right ventricular deformation is estimated from tagged cardiac magnetic resonance imaging using volumetric deformable models constructed from nonuniform rational B-splines (NURBS). The four model types considered and compared for the left ventricle include two Cartesian NURBS models--one with a cylindrical parameter assignment and one with a prolate spheroidal parameter assignment. The remaining two are non-Cartesian, i.e., prolate spheroidal and cylindrical each with their respective prolate spheroidal and cylindrical parameter assignment regimes. These choices were made based on the typical shape of the left ventricle. For each frame starting with end-diastole, a NURBS model is constructed by fitting two surfaces with the same parameterization to the corresponding set of epicardial and endocardial contours from which a volumetric model is created. Using normal displacements of the three sets of orthogonal tag planes as well as displacements of contour/tag line intersection points and tag plane intersection points, one can solve for the optimal homogeneous coordinates, in a weighted least squares sense, of the control points of the deformed NURBS model at end-diastole using quadratic programming. This allows for subsequent nonrigid registration of the biventricular model at end-diastole to all later time frames. After registration of the model to all later time points, the registered NURBS models are temporally lofted in order to create a comprehensive four-dimensional NURBS model. From the lofted model, we can extract three-dimensional myocardial deformation fields and corresponding Lagrangian and Eulerian strain maps which are local measures of nonrigid deformation. The results show that, in the case of simulated data, the quadratic Cartesian NURBS models with the cylindrical and prolate spheroidal parameter assignments outperform their counterparts in predicting normal strain. The decreased complexity associated with the Cartesian model with the cylindrical parameter assignment prompted its use for subsequent calculations. Lagrangian strains in three canine data, a normal human, and a patient with history of myocardial infarction are presented. Eulerian strains for the normal human data are also included.

The aim of this paper is to present a hybrid approach to accurate quantification of vascular structures from magnetic resonance angiography (MRA) images using level set methods and deformable geometric models constructed with 3-D Delaunay triangulation. Multiple scale filtering based on the analysis of local intensity structure using the Hessian matrix is used to effectively enhance vessel structures with various diameters. The level set method is then applied to automatically segment vessels enhanced by the filtering with a speed function derived from enhanced MRA images. Since the goal of this paper is to obtain highly accurate vessel borders, suitable for use in fluid flow simulations, in a subsequent step, the vessel surface determined by the level set method is triangulated using 3-D Delaunay triangulation and the resulting surface is used as a parametric deformable model. Energy minimization is then performed within a variational setting with a first-order internal energy; the external energy is derived from 3-D image gradients. Using the proposed method, vessels are accurately segmented from MRA data.

Magnetic resonance angiography (MRA) has become an important tool for the clinical evaluation of vascular disease. Flow measurement with phase-contrast (PC) magnetic resonance (MR) imaging provides a powerful method for evaluation of blood velocity information inside vessels. However, image artifacts from complex flow patterns including slow flow, recirculation zone, and pulsatile flow can adversely affect accuracy of results. In this paper, we introduce a new numerical formulation for improving the accuracy of PC velocity fields and corresponding streamlines, based on a physical constraint from fluid dynamics, within a regularization framework. The formulation which makes use of a stream function, automatically enforces continuity constraint of incompressible flow and reconstructs the flow streamlines from PC images. We applied the algorithm to complex MR imaging flow velocities obtained in a flow phantom of an axisymmetric abdominal aortic aneurysm. The algorithm significantly improved streamline results especially inside the recirculation zone, where artifacts are more pronounced. A velocity reconstruction method in primitive variable form is also presented and results are compared with the stream function method. In order to validate flow characteristics derived from PC MR images, we used the FLUENT computational fluid dynamics software package, to simulate flow patterns within the same geometry as our phantom. There was a good agreement between the numerical simulations and recovered PC streamline results. Processed streamlines, in both stream function and primitive variable methods, were more realistic and provided more precise flow patterns than unprocessed PC data. Additionally, the feasibility of the method was demonstrated in the aorta of a normal volunteer.

Magnetic resonance (MR) tagging is a technique for measuring heart deformations through creation of a stripe grid pattern on cardiac images. In this paper, we present a maximum a posteriori (MAP) framework for detecting tag lines using a Markov random field (MRF) defined on the lattice generated by three-dimensional (3-D) and four-dimensional (4-D) (3-D + t) uniform sampling of B-spline models. In the 3-D case, MAP estimation is cast for detecting present tag features in the current image given an initial solid from the previous frame (the initial undeformed solid is manually positioned by clicking on corner points of a cube). The method also allows the parameters of the solid model, including the number of knots and the spline order, to be adjusted within the same framework. Fitting can start with a solid with less knots and lower spline order and proceed to one with more knots and/or higher order so as to achieve more accuracy and/or higher order of smoothness. In the 4-D case, the initial model is considered to be the linear interpolation of a sequence of optimal solids obtained from 3-D tracking. The same framework proposed for the 3-D case can once again be applied to arrive at a 4-D B-spline model with a higher temporal order.

Magnetic resonance imaging (MRI) is unique in its ability to noninvasively and selectively alter tissue magnetization, and create tag planes intersecting image slices. The resulting grid of signal voids allows for tracking deformations of tissues in otherwise homogeneous-signal myocardial regions. In this paper, we propose a specific spatial modulation of magnetization (SPAMM) imaging protocol together with efficient techniques for measurement of three-dimensional (3-D) motion of material points of the human heart (referred to as myocardial beads) from images collected with the SPAMM method. The techniques make use of tagged images in orthogonal views by explicitly reconstructing 3-D B-spline surface representation of tag planes (tag planes in two orthogonal orientations intersecting the short-axis (SA) image slices and tag planes in an orientation orthogonal to the short-axis tag planes intersecting long-axis (LA) image slices). The developed methods allow for viewing deformations of 3-D tag surfaces, spatial correspondence of long-axis and short-axis image slice and tag positions, as well as nonrigid movement of myocardial beads as a function of time.

Cardiac motion estimation is very important in understanding cardiac dynamics and in noninvasive diagnosis of heart disease. Magnetic resonance (MR) imaging tagging is a technique for measuring heart deformations. In cardiac tagged MR images, a set of dark lines are noninvasively encoded within myocardial tissue providing the means for measurement of deformations of the heart. The points along tag lines measured in different frames and in different directions carry important information for determining the three-dimensional nonrigid movement of left ventricle. However, these measurements are sparse and, therefore, multidimensional interpolation techniques are needed to reconstruct a dense displacement field. In this paper, a novel subspace approximation technique is used to accomplish this task. We formulate the displacement estimation as a variational problem and then project the solution into spline subspaces. Efficient numerical methods are derived by taking advantages of B-spline properties. The proposed technique significantly improves our previous results reported in [3] with respect to computational time. The method is applied to a temporal sequence of two-dimensional images and is validated with simulated and in vivo heart data.

Accurate delineation of the volumetric motion of the left ventricle (LV) of the heart from tagged magnetic resonance imaging (MRI) is an important area of research. We have built a system that takes extracted tag line features from short axis (SA) and long axis (LA) image sequences as input and fits a four-dimensional (4-D) time-varying B-spline model to the data by simultaneously fitting the model knot solids to MRI frames via matching three sequences of solid knot planes to the LV tag planes for 4-D tracking. Important advantages of the model are that reconstruction of tag surfaces, three-dimensional (3-D) material point localization, as well as displacement reconstruction are all achieved in a single step. The generated 3-D displacement fields are validated with a cardiac motion simulator, and 3-D motion fields capturing in vivo deformations in a porcine model with posterolateral myocardial infarction are illustrated.

In this invited paper, an overview of techniques developed at the Cardiovascular Image Analysis Laboratory at Washington University is discussed. At the core of the authors' methodologies lie flexible shape models, which are employed in automated as well as semi-automated analysis of cardiac MRI and X-ray angiography images. The mathematical bases used for the flexible templates are of the B-spline variety, providing compact representation and interactive capabilities for manipulation of curves, surfaces, and volumes.

Accurate delineation of anatomical objects in 3D volumetric data is a significant problem in medical imaging. The authors have built a system that takes as input a spatial stack of 2D image slices being studied. The output of the system is a smooth 3D surface delineating the medical structure of interest, and the volume enclosed by the 3D surface. This paper presents a new method to represent a 3D anatomical object by a deformable B-spline tube, that facilitates the calculation of the volume enclosed based on a closed-form volume calculation formula. In order to validate the deformable B-spline surface fitting and associated volume calculation, 19 simulated 3D images were considered and theoretical volumes were compared with volumes produced by the authors' system. The errors were less than 4.9%.

Magnetic resonance imaging (MRI) is unique in its ability to noninvasively and selectively alter tissue magnetization and create tagged patterns within a deforming body such as the heart muscle. The resulting patterns define a time-varying curvilinear coordinate system on the tissue, which we track with coupled B-snake grids. B-spline bases provide local control of shape, compact representation, and parametric continuity. Efficient spline warps are proposed which warp an area in the plane such that two embedded snake grids obtained from two tagged frames are brought into registration, interpolating a dense displacement vector field. The reconstructed vector field adheres to the known displacement information at the intersections, forces corresponding snakes to be warped into one another, and for all other points in the plane, where no information is available, a C1 continuous vector field is interpolated. The implementation proposed in this paper improves on our previous variational-based implementation and generalizes warp methods to include biologically relevant contiguous open curves, in addition to standard landmark points. The methods are validated with a cardiac motion simulator, in addition to in-vivo tagging data sets.

Although current edge-following schemes can be very efficient in determining coronary boundaries, they may fail when the feature to be followed is disconnected (and the scheme is unable to bridge the discontinuity) or branch points exist where the best path to follow is indeterminate. In this paper, we present new deformable spline algorithms for determining vessel boundaries, and enhancing their centerline features. A bank of even and odd S-Gabor filter pairs of different orientations are convolved with vascular images in order to create an external snake energy field. Each filter pair will give maximum response to the segment of vessel having the same orientation as the filters. The resulting responses across filters of different orientations are combined to create an external energy field for snake optimization. Vessels are represented by B-Spline snakes, and are optimized on filter outputs with dynamic programming. The points of minimal constriction and the percent-diameter stenosis are determined from a computed vessel centerline. The system has been statistically validated using fixed stenosis and flexible-tube phantoms. It has also been validated on 20 coronary lesions with two independent operators, and has been tested for interoperator and intraoperator variability and reproducibility. The system has been found to be specially robust in complex images involving vessel branchings and incomplete contrast filling.

Dynamic programming is discussed as an approach to solving variational problems in vision. Dynamic programming ensures global optimality of the solution, is numerically stable, and allows for hard constraints to be enforced on the behavior of the solution within a natural and straightforward structure. As a specific example of the approach's efficacy, applying dynamic programming to the energy-minimizing active contours is described. The optimization problem is set up as a discrete multistage decision process and is solved by a time-delayed discrete dynamic programming algorithm. A parallel procedure for decreasing computational costs is discussed.>

Energy-Minimizing Active Contour Models (snakes) have recently been proposed by Kass et al. [8] as a top-down mechanism for locating features of interest in images. The Kass et al.’s algorithm involves four steps: setting up a variational integral on the continuous plane, deriving a pair of Euler equations, discretizing them, and solving the discrete equations iteratively until convergence. This algorithm suffers from a number of problems. We discuss these problems and present an algoIithm for active contours based on dynamic programming. The optimization problem is set up as a discrete multi-stage decision process and is solved by a “time-delayed” discrete dynamic programming algorithm. This formulation leads to a stable behavior for the active contours over iterations, in addition to allowing for hard constraints to be enforced on the behavior of the solution. Results of the application of the proposed algorithm to real images is presented.