Esophageal cancer is categorized as a type of disease with a high mortality rate. Early detection of esophageal abnormalities (i.e. precancerous and early cancerous) can improve the survival rate of the patients. Recent deep learning-based methods for selected types of esophageal abnormality detection from endoscopic images have been proposed. However, no methods have been introduced in the literature to cover the detection from endoscopic videos, detection from challenging frames and detection of more than one esophageal abnormality type. In this paper, we present an efficient method to automatically detect different types of esophageal abnormalities from endoscopic videos. We propose a novel 3D Sequential DenseConvLstm network that extracts spatiotemporal features from the input video. Our network incorporates 3D Convolutional Neural Network (3DCNN) and Convolutional Lstm (ConvLstm) to efficiently learn short and long term spatiotemporal features. The generated feature map is utilized by a region proposal network and ROI pooling layer to produce a bounding box that detects abnormality regions in each frame throughout the video. Finally, we investigate a post-processing method named Frame Search Conditional Random Field (FS-CRF) that improves the overall performance of the model by recovering the missing regions in neighborhood frames within the same clip. We extensively validate our model on an endoscopic video dataset that includes a variety of esophageal abnormalities. Our model achieved high performance using different evaluation metrics showing 93.7% recall, 92.7% precision, and 93.2% F-measure. Moreover, as no results have been reported in the literature for the esophageal abnormality detection from endoscopic videos, to validate the robustness of our model, we have tested the model on a publicly available colonoscopy video dataset, achieving the polyp detection performance in a recall of 81.18%, precision of 96.45% and F-measure 88.16%, compared to the state-of-the-art results of 78.84% recall, 90.51% precision and 84.27% F-measure using the same dataset. This demonstrates that the proposed method can be adapted to different gastrointestinal endoscopic video applications with a promising performance.

Lesion border detection is considered a crucial step in diagnosing skin cancer. However, performing such a task automatically is challenging due to the low contrast between the surrounding skin and lesion, ambiguous lesion borders, and the presence of artifacts such as hair. In this paper we propose a two-stage approach for skin lesion border detection: (i) segmenting the skin lesion dermoscopy image using U-Net, and (ii) extracting the edges from the segmented image using a novel approach we call FuzzEdge. The proposed approach is compared with another published skin lesion border detection approach, and the results show that our approach performs better in detecting the main borders of the lesion and is more robust to artifacts that might be present in the image. The approach is also compared with the manual border drawings of a dermatologist, resulting in an average Dice similarity of 87.7%.

There exist various types of information in retail food packages, including food product name, ingredients list and use by date. The correct recognition and coding of use by dates is especially critical in ensuring proper distribution of the product to the market and eliminating potential health risks caused by erroneous mislabelling. The latter can have a major negative effect on the health of consumers and consequently raise legal issues for suppliers. In this work, an end-to-end architecture, composed of a dual deep neural network based system is proposed for automatic recognition of use by dates in food package photos. The system includes: a Global level convolutional neural network (CNN) for high-level food package image quality evaluation (blurry/clear/missing use by date statistics); a Local level fully convolutional network (FCN) for use by date ROI localisation. Post ROI extraction, the date characters are then segmented and recognised. The proposed framework is the first to employ deep neural networks for end-to-end automatic use by date recognition in retail packaging photos. It is capable of achieving very good levels of performance on all the aforementioned tasks, despite the varied textual/pictorial content complexity found in food packaging design.

Compressed sensing magnetic resonance imaging (CS-MRI) enables fast acquisition, which is highly desirable for numerous clinical applications. This can not only reduce the scanning cost and ease patient burden, but also potentially reduce motion artefacts and the effect of contrast washout, thus yielding better image quality. Different from parallel imaging-based fast MRI, which utilizes multiple coils to simultaneously receive MR signals, CS-MRI breaks the Nyquist-Shannon sampling barrier to reconstruct MRI images with much less required raw data. This paper provides a deep learning-based strategy for reconstruction of CS-MRI, and bridges a substantial gap between conventional non-learning methods working only on data from a single image, and prior knowledge from large training data sets. In particular, a novel conditional Generative Adversarial Networks-based model (DAGAN)-based model is proposed to reconstruct CS-MRI. In our DAGAN architecture, we have designed a refinement learning method to stabilize our U-Net based generator, which provides an end-to-end network to reduce aliasing artefacts. To better preserve texture and edges in the reconstruction, we have coupled the adversarial loss with an innovative content loss. In addition, we incorporate frequency-domain information to enforce similarity in both the image and frequency domains. We have performed comprehensive comparison studies with both conventional CS-MRI reconstruction methods and newly investigated deep learning approaches. Compared with these methods, our DAGAN method provides superior reconstruction with preserved perceptual image details. Furthermore, each image is reconstructed in about 5 ms, which is suitable for real-time processing.

In this paper, we introduce a diagnosis and treatment for amblyopia performed through a game suitable for children aged between 3 and 7. Our method places emphasis on cooperation between the two eyes to achieve a good binocular outcome to aid the recovery of depth perception. Our approach is not limited to a particular device or platform nor even to a aprticular form of game. Several prototype games have been developed, including 2D games and 3D games.

Abstract The aim of removing camera shake is to estimate a sharp version x from a shaken image y when the blur kernel k is unknown. Recent research on this topic evolved through two paradigms called and . only solves for k by marginalizing the image prior, while recovers both x and k by selecting the mode of the posterior distribution. This paper first systematically analyses the latent limitations of these two estimators through Bayesian analysis. We explain the reason why it is so difficult for image statistics to solve the previously reported failure. Then we show that the leading methods, which depend on efficient prediction of large step edges, are not robust to natural images due to the diversity of edges. , although much more robust to diverse edges, is constrained by two factors: the prior variation over different images, and the ratio between image size and kernel size. To overcome these limitations, we introduce an inter‐scale prior prediction scheme and a principled mechanism for integrating the sharpening filter into . Both qualitative results and extensive quantitative comparisons demonstrate that our algorithm outperforms state‐of‐the‐art methods.

Blind motion deblurring estimates a sharp image from a motion blurred image without the knowledge of the blur kernel. Although significant progress has been made on tackling this problem, existing methods, when applied to highly diverse natural images, are still far from stable. This paper focuses on the robustness of blind motion deblurring methods toward image diversity-a critical problem that has been previously neglected for years. We classify the existing methods into two schemes and analyze their robustness using an image set consisting of 1.2 million natural images. The first scheme is edge-specific, as it relies on the detection and prediction of large-scale step edges. This scheme is sensitive to the diversity of the image edges in natural images. The second scheme is nonedge-specific and explores various image statistics, such as the prior distributions. This scheme is sensitive to statistical variation over different images. Based on the analysis, we address the robustness by proposing a novel nonedge-specific adaptive scheme (NEAS), which features a new prior that is adaptive to the variety of textures in natural images. By comparing the performance of NEAS against the existing methods on a very large image set, we demonstrate its advance beyond the state-of-the-art.

This paper presents an efficient algorithm for segmenting different types of pulmonary nodules including high and low contrast nodules, nodules with vasculature attachment, and nodules in the close vicinity of the lung wall or diaphragm. The algorithm performs an adaptive sphericity oriented contrast region growing on the fuzzy connectivity map of the object of interest. This region growing is operated within a volumetric mask which is created by first applying a local adaptive segmentation algorithm that identifies foreground and background regions within a certain window size. The foreground objects are then filled to remove any holes, and a spatial connectivity map is generated to create a 3-D mask. The mask is then enlarged to contain the background while excluding unwanted foreground regions. Apart from generating a confined search volume, the mask is also used to estimate the parameters for the subsequent region growing, as well as for repositioning the seed point in order to ensure reproducibility. The method was run on 815 pulmonary nodules. By using randomly placed seed points, the approach was shown to be fully reproducible. As for acceptability, the segmentation results were visually inspected by a qualified radiologist to search for any gross miss-segmentation. 84% of the first results of the segmentation were accepted by the radiologist while for the remaining 16% nodules, alternative segmentation solutions that were provided by the method were selected.

Coronary artery calcification (CAC) is quantified based on a computed tomography (CT) scan image. A calcified region is identified. Modified expectation maximization (MEM) of a statistical model for the calcified and background material is used to estimate the partial calcium content of the voxels. The algorithm limits the region over which MEM is performed. By using MEM, the statistical properties of the model are iteratively updated based on the calculated resultant calcium distribution from the previous iteration. The estimated statistical properties are used to generate a map of the partial calcium content in the calcified region. The volume of calcium in the calcified region is determined based on the map. The experimental results on a cardiac phantom, scanned 90 times using 15 different protocols, demonstrate that the proposed method is less sensitive to partial volume effect and noise, with average error of 9.5% (standard deviation (SD) of 5-7mm(3)) compared with 67% (SD of 3-20mm(3)) for conventional techniques. The high reproducibility of the proposed method for 35 patients, scanned twice using the same protocol at a minimum interval of 10 min, shows that the method provides 2-3 times lower interscan variation than conventional techniques.

This paper presents a new method for shape based segmentation of 3D medical images. 3D geometric information is calculated for each voxel by computing the partial derivatives of the 3D image. The shape features of the iso-intensity surfaces are subsequently extracted. The extracted shape features are combined with 3D intensity-based region growing to give accurate separation of connected objects having different shapes but similar intensity values. We have applied this method to both synthetic and real 3D CT lung images. The experimental results demonstrate that the presented method, unlike the traditional intensity-based method, is able to segment connected objects accurately. A sphere can be differentiated from a connected cylindrical shape within the synthetic data. In the case of the real 3D CT lung images, all of the nodules can be detected and separated accurately from adjoining blood vessel or from the lung wall.

A new method is proposed to reconstruct and analyze the left ventricle (LV) from multiple acoustic window three-dimensional (3-D) ultrasound acquired using a transthoracic 3-D rotational probe. Prior research in this area has been based on one acoustic window acquisition. However, the data suffers from several limitations that degrade the reconstruction and reduce the clinical value of interpretation, such as the presence of shadow due to bone (ribs) and air (in the lungs) and motion of the probe during the acquisition. In this paper, we show how to overcome these limitations by automatically fusing information from multiple acoustic window sparse-view acquisitions and using a position sensor to track the probe in real time. Geometric constraints of the object shape, and spatiotemporal information relating to the image acquisition process, are used in new algorithms for 1) grouping endocardial edge cues from an initial image segmentation and 2) defining a novel reconstruction method that utilizes information from multiple acoustic windows. The new method has been validated on a phantom and three real heart data sets. In the phantom study, one finger of a latex glove was scanned from two acoustic windows and reconstructed using the new method. The volume error was measured to be less than 4%. In the clinical case study, 3-D ultrasound and magnetic resonance imaging (MRI) scanning were performed on the same healthy volunteers. Quantitative ejection fractions (EFs) and volume-time curves over a cardiac cycle were estimated using the new method and compared to cardiac MRI measurements. This showed that the new method agrees better with MRI measurements than the previous approach we have developed based on a single acoustic window. The EF errors of the new method with respect to MRI measurements were less than 6%. A more extensive clinical validation is required to establish whether these promising first results translate to a method suitable for routine clinical use.