Visual servoing for the development of autonomous robotic systems capable of administering UltraSound (US) guided regional anesthesia requires real-time segmentation of nerves, needle tip localization and needle trajectory extrapolation. First, we recruited 227 patients to build a large dataset of 41,000 anesthesiologist annotated images from US videos of brachial plexus nerves and developed models to localize nerves in the US images. Generalizability of the best suited model was tested on the datasets constructed from separate US scanners. Using these nerve segmentation predictions, we define automated anesthesia needle targets by fitting an ellipse to the nerve contours. Next, we developed an image analysis tool to guide the needle toward their targets. For the segmentation of the needle, a natural RGB pre-trained neural network was first fine-tuned on a large US dataset for domain transfer and then adapted for the needle using a small dataset. The segmented needle’s trajectory angle is calculated using Radon transformation and the trajectory is extrapolated from the needle tip. The intersection of the extrapolated trajectory with the needle target guides the needle navigation for drug delivery. The needle trajectory’s average error was within acceptable range of 5 mm as per experienced anesthesiologists. The entire dataset has been released publicly for further study by the research community at https://github.com/Regional-US/

Biological processes like growth, aging, and disease progression are generally studied with follow-up scans taken at different time points, i.e., image time series (TS) based analysis. Image TS represents the evolution of anatomy over time, but different anatomies may have different structural characteristics and temporal paths. Therefore, separating the time-dependent path difference and time-independent basic anatomy/shape changes is important when comparing two image TS to understand the causes of the observed differences better. A method to untangle and quantify the path and shape difference between the TS is presented in this paper. The proposed method is evaluated with simulated and adult and fetal neuro templates. Results show that the metric can separate and quantify the path and shape differences between TS.

Early detection and treatment of glaucoma is of interest as it is a chronic eye disease leading to an irreversible loss of vision. Existing automated systems rely largely on fundus images for assessment of glaucoma due to their fast acquisition and cost-effectiveness. Optical Coherence Tomographic ( OCT ) images provide vital and unambiguous information about nerve fiber loss and optic cup morphology, which are essential for disease assessment. However, the high cost of OCT is a deterrent for deployment in screening at large scale. In this article, we present a novel CAD solution wherein both OCT and fundus modality images are leveraged to learn a model that can perform a mapping of fundus to OCT feature space. We show how this model can be subsequently used to detect glaucoma given an image from only one modality (fundus). The proposed model has been validated extensively on four public andtwo private datasets. It attained an AUC/Sensitivity value of 0.9429/0.9044 on a diverse set of 568 images, which is superior to the figures obtained by a model that is trained only on fundus features. Cross-validation was also done on nearly 1,600 images drawn from a private (OD-centric) and a public (macula-centric) dataset and the proposed model was found to outperform the state-of-the-art method by 8% (public) to 18% (private). Thus, we conclude that fundus to OCT feature space mapping is an attractive option for glaucoma detection.

Deep learning based methods have shown great promise in achieving accurate automatic detection of Coronavirus Disease (covid) - 19 from Chest X-Ray (cxr) images.However, incorporating explainability in these solutions remains relatively less explored. We present a hierarchical classification approach for separating normal, non-covid pneumonia (ncp) and covid cases using cxr images. We demonstrate that the proposed method achieves clinically consistent explainations. We achieve this using a novel multi-scale attention architecture called Multi-scale Attention Residual Learning (marl) and a new loss function based on conicity for training the proposed architecture. The proposed classification strategy has two stages. The first stage uses a model derived from DenseNet to separate pneumonia cases from normal cases while the second stage uses the marl architecture to discriminate between covid and ncp cases. With a five-fold cross validation the proposed method achieves 93%, 96.28%, and 84.51% accuracy respectively over three large, public datasets for normal vs. ncp vs. covid classification. This is competitive to the state-of-the-art methods. We also provide explanations in the form of GradCAM attributions, which are well aligned with expert annotations. The attributions are also seen to clearly indicate that marl deems the peripheral regions of the lungs to be more important in the case of covid cases while central regions are seen as more important in ncp cases. This observation matches the criteria described by radiologists in clinical literature, thereby attesting to the utility of the derived explanations.

Disease diagnosis often requires segmentation of structures from a given image followed by shape analysis. Shape analysis entails quantifying the variability in a shape by constructing a template for a given population. We propose an orientation-invariant representation using varifolds for the shape elements in a given shape population and present a novel diffeomorphic Log-demons based template creation pipeline. The proposed method generates a good quality template at a significantly less computation time compared to state of the art method.

The level set based deformable models (LDM) are commonly used for medical image segmentation. However, they rely on a handcrafted curve evolution velocity that needs to be adapted for each segmentation task. The Convolutional Neural Networks (CNN) address this issue by learning robust features in a supervised end-to-end manner. However, CNNs employ millions of network parameters, which require a large amount of data during training to prevent over-fitting and increases the memory requirement and computation time during testing. Moreover, since CNNs pose segmentation as a region-based pixel labeling, they cannot explicitly model the high-level dependencies between the points on the object boundary to preserve its overall shape, smoothness or the regional homogeneity within and outside the boundary. We present a Recurrent Neural Network based solution called the RACE-net to address the above issues. RACE-net models a generalized LDM evolving under a constant and mean curvature velocity. At each time-step, the curve evolution velocities are approximated using a feed-forward architecture inspired by the multiscale image pyramid. RACE-net allows the curve evolution velocities to be learned in an end-to-end manner while minimizing the number of network parameters, computation time, and memory requirements. The RACE-net was validated on three different segmentation tasks: optic disc and cup in color fundus images, cell nuclei in histopathological images, and the left atrium in cardiac MRI volumes. Assessment on public datasets was seen to yield high Dice values between 0.87 and 0.97, which illustrates its utility as a generic, off-the-shelf architecture for biomedical segmentation.

Automated and accurate segmentation of cystoid structures in optical coherence tomography (OCT) is of interest in the early detection of retinal diseases. It is, however, a challenging task. We propose a novel method for localizing cysts in 3-D OCT volumes. The proposed work is biologically inspired and based on selective enhancement of the cysts, by inducing motion to a given OCT slice. A convolutional neural network is designed to learn a mapping function that combines the result of multiple such motions to produce a probability map for cyst locations in a given slice. The final segmentation of cysts is obtained via simple clustering of the detected cyst locations. The proposed method is evaluated on two public datasets and one private dataset. The public datasets include the one released for the OPTIMA cyst segmentation challenge (OCSC) in MICCAI 2015 and the DME dataset. After training on the OCSC train set, the method achieves a mean dice coefficient (DC) of 0.71 on the OCSC test set. The robustness of the algorithm was examined by cross validation on the DME and AEI (private) datasets and a mean DC values obtained were 0.69 and 0.79, respectively. Overall, the proposed system has the highest performance on all the benchmarks. These results underscore the strengths of the proposed method in handling variations in both data acquisition protocols and scanners.

Retinal swelling due to the accumulation of fluid is associated with the most vision-threatening retinal diseases. Optical coherence tomography (OCT) is the current standard of care in assessing the presence and quantity of retinal fluid and image-guided treatment management. Deep learning methods have made their impact across medical imaging, and many retinal OCT analysis methods have been proposed. However, it is currently not clear how successful they are in interpreting the retinal fluid on OCT, which is due to the lack of standardized benchmarks. To address this, we organized a challenge RETOUCH in conjunction with MICCAI 2017, with eight teams participating. The challenge consisted of two tasks: fluid detection and fluid segmentation. It featured for the first time: all three retinal fluid types, with annotated images provided by two clinical centers, which were acquired with the three most common OCT device vendors from patients with two different retinal diseases. The analysis revealed that in the detection task, the performance on the automated fluid detection was within the inter-grader variability. However, in the segmentation task, fusing the automated methods produced segmentations that were superior to all individual methods, indicating the need for further improvements in the segmentation performance.

In this work, we propose an online handwriting solution, where the data is captured with the help of depth sensors. Users may write in the air and our method recognizes it in real time using the proposed feature representation. Our method uses an efficient fingertip tracking approach and reduces the necessity of pen-up/pen-down switching. We validate our method on two depth sensors, Kinect and Leap Motion Controller. On a dataset collected from 20 users, we achieve a recognition accuracy of 97.59% for character recognition. We also demonstrate how this system can be extended for lexicon recognition with reliable performance. We have also prepared a dataset containing 1,560 characters and 400 words with the intention of providing common benchmark for handwritten character recognition using depth sensors and related research.

Nuclear imaging modalities like Positron emission tomography (PET) are characterized by a low SNR value due to the underlying signal generation mechanism. Given the significant role images play in current-day diagnostics, obtaining noise-free PET images is of great interest. With its higher packing density and larger and symmetrical neighbourhood, the hexagonal lattice offers a natural robustness to degradation in signal. Based on this observation, we propose an alternate solution to denoising, namely by changing the sampling lattice. We use filtered back projection for reconstruction, followed by a sparse dictionary based denoising and compare noise-free reconstruction on the Square and Hexagonal lattices. Experiments with PET phantoms (NEMA, Hoffman) and the Shepp-Logan phantom show that the improvement in denoising, post reconstruction, is not only at the qualitative but also quantitative level. The improvement in PSNR in the hexagonal lattice is on an average between 2 to 10 dB. These results establish the potential of the hexagonal lattice for reconstruction from noisy data, in general.

Diabetic macular edema (DME) is an advanced symptom of diabetic retinopathy and can lead to irreversible vision loss. In this paper, a two-stage methodology for the detection and classification of DME severity from color fundus images is proposed. DME detection is carried out via a supervised learning approach using the normal fundus images. A feature extraction technique is introduced to capture the global characteristics of the fundus images and discriminate the normal from DME images. Disease severity is assessed using a rotational asymmetry metric by examining the symmetry of macular region. The performance of the proposed methodology and features are evaluated against several publicly available datasets. The detection performance has a sensitivity of 100% with specificity between 74% and 90%. Cases needing immediate referral are detected with a sensitivity of 100% and specificity of 97%. The severity classification accuracy is 81% for the moderate case and 100% for severe cases. These results establish the effectiveness of the proposed solution.

Automatic retinal image analysis is emerging as an important screening tool for early detection of eye diseases. Glaucoma is one of the most common causes of blindness. The manual examination of optic disk (OD) is a standard procedure used for detecting glaucoma. In this paper, we present an automatic OD parameterization technique based on segmented OD and cup regions obtained from monocular retinal images. A novel OD segmentation method is proposed which integrates the local image information around each point of interest in multidimensional feature space to provide robustness against variations found in and around the OD region. We also propose a novel cup segmentation method which is based on anatomical evidence such as vessel bends at the cup boundary, considered relevant by glaucoma experts. Bends in a vessel are robustly detected using a region of support concept, which automatically selects the right scale for analysis. A multi-stage strategy is employed to derive a reliable subset of vessel bends called r-bends followed by a local spline fitting to derive the desired cup boundary. The method has been evaluated on 138 images comprising 33 normal and 105 glaucomatous images against three glaucoma experts. The obtained segmentation results show consistency in handling various geometric and photometric variations found across the dataset. The estimation error of the method for vertical cup-to-disk diameter ratio is 0.09/0.08 (mean/standard deviation) while for cup-to-disk area ratio it is 0.12/0.10. Overall, the obtained qualitative and quantitative results show effectiveness in both segmentation and subsequent OD parameterization for glaucoma assessment.

In this paper, we present a method for cup boundary detection from monocular colour fundus image to help quantify cup changes. The method is based on anatomical evidence such as vessel bends at cup boundary, considered relevant by glaucoma experts. Vessels are modeled and detected in a curvature space to better handle inter-image variations. Bends in a vessel are robustly detected using a region of support concept, which automatically selects the right scale for analysis. A reliable subset called r-bends is derived using a multi-stage strategy and a local spline fitting is used to obtain the desired cup boundary. The method has been successfully tested on 133 images comprising 32 normal and 101 glaucomatous images against three glaucoma experts. The proposed method shows high sensitivity in cup to disk ratio-based glaucoma detection and local assessment of the detected cup boundary shows good consensus with the expert markings.

The ability to detect, and track multiple moving objects like person and other robots, is an important prerequisite for mobile robots working in dynamic indoor environments. We approach this problem by detecting independently moving objects in image sequence from a monocular camera mounted on a robot. We use multi-view geometric constraints to classify a pixel as moving or static. The first constraint, we use, is the epipolar constraint which requires images of static points to lie on the corresponding epipolar lines in subsequent images. In the second constraint, we use the knowledge of the robot motion to estimate a bound in the position of image pixel along the epipolar line. This is capable of detecting moving objects followed by a moving camera in the same direction, a so-called degenerate configuration where the epipolar constraint fails. To classify the moving pixels robustly, a Bayesian framework is used to assign a probability that the pixel is stationary or dynamic based on the above geometric properties and the probabilities are updated when the pixels are tracked in subsequent images. The same framework also accounts for the error in estimation of camera motion. Successful and repeatable detection and pursuit of people and other moving objects in realtime with a monocular camera mounted on the Pioneer 3DX, in a cluttered environment confirms the efficacy of the method.

Extraction of anatomical structures (landmarks), such as optic disk (OD), fovea and blood vessels, from fundus images is use-ful in automatic diagnosis. Current approaches largely use spa-tial relationship among the landmarks ’ position for detection. In this paper, we present an appearance-based method for de-tecting fovea and OD from colour images. The strategy used for detection is based on improving the local contrast which is achieved by combining information from two spectral channels of the given image. The proposed method has been successfully tested on different datasets and the results show 96 % detection for fovea and 91 % detection for OD (a total of 502 and 531 im-ages for fovea and OD are taken respectively). Index Terms — colour retinal image,contrast enhancement, fovea/optic disk detection 1.

A 2D function, representing a digital image, is a surface in 3D space. Curvature of such a surface can be exploited to detect ridge and valley like features from images. In this paper, we present an analysis of such curvature based ridge and valley detection techniques and come up with a description for the different classes of ridge and valley profiles which can be detected by them. Such an analysis helps in understanding the scope and limitations of the curvature based techniques. As curvature is a measure of 'bend' in the cross-section profile along a particular direction of the image intensities, the analysis is presented using ID functions which represent cross-section profiles of ridges and valleys. The classes of profiles which can be detected by a curvature based technique are described in terms of the properties of the second-derivative of the ID profile function