Distributed big data and digital healthcare technologies have great potential to promote medical services, but challenges arise when it comes to learning predictive model from diverse and complex e-health datasets. Federated Learning (FL), as a collaborative machine learning technique, aims to address the challenges by learning a joint predictive model across multi-site clients, especially for distributed medical institutions or hospitals. However, most existing FL methods assume that clients possess fully labeled data for training, which is often not the case in e-health datasets due to high labeling costs or expertise requirement. Therefore, this work proposes a novel and feasible approach to learn a Federated Semi-Supervised Learning (FSSL) model from distributed medical image domains, where a federated pseudo-labeling strategy for unlabeled clients is developed based on the embedded knowledge learned from labeled clients. This greatly mitigates the annotation deficiency at unlabeled clients and leads to a cost-effective and efficient medical image analysis tool. We demonstrated the effectiveness of our method by achieving significant improvements compared to the state-of-the-art in both fundus image and prostate MRI segmentation tasks, resulting in the highest Dice scores of 89.23% and 91.95% respectively even with only a few labeled clients participating in model training. This reveals the superiority of our method for practical deployment, ultimately facilitating the wider use of FL in healthcare and leading to better patient outcomes.

Mitosis detection is one of the challenging steps in biomedical imaging research, which can be used to observe the cell behavior. Most of the already existing methods that are applied in detecting mitosis usually contain many nonmitotic events (normal cell and background) in the result (false positives, FPs). In order to address such a problem, in this study, we propose to apply 2.5-dimensional (2.5D) networks called CasDetNet_CLSTM, which can accurately detect mitotic events in 4D microscopic images. This CasDetNet_CLSTM involves a 2.5D faster region-based convolutional neural network (Faster R-CNN) as the first network, and a convolutional long short-term memory (CLSTM) network as the second network. The first network is used to select candidate cells using the information from nearby slices, whereas the second network uses temporal information to eliminate FPs and refine the result of the first network. Our experiment shows that the precision and recall of our networks yield better results than those of other state-of-the-art methods.

Mitotic event detection is a fundamental step in investigating of cell behaviors. The event can be used to analyze various diseases, but most mitotic event detections performed previously focused only on two-dimensional (2D) images with time information. Owing to the complex background (normal cells) and mitotic event orientations, the 2D detection methods yield many false positive and false negative results. To solve this problem, we proposed a 2.5 dimensional (2.5D) cascaded end-to-end network combined with 3D anchors for accurate detection of mitotic events in 4D microscopic images. Our proposed network uses a convolutional long short-term memory to handle issues relating to time sequence; this helps to improve the detection accuracy (reduction of false positives). Furthermore, it uses 3D anchors to capture volume information used to address the orientation problem (reduction of false negatives). The experimental results show that the proposed method can achieve higher precision and recall compared with state-of-the-art methods.

Detecting, localizing and counting small circular objects in machine parts is an important task in many applications for manufacturing. Existing methods of circle detection face difficulties due to the high-curvature and limited edge points of circles. As a result, in this paper we propose a novel two-stage circle detection method, which integrates bottom-up coarse detection and top-down circle fitting. First, a circle detector combining low-level feature descriptors and a linear SVM is developed. This is used to scan an input image in a sliding window mode to detect small circles with coarse estimates of locations and scales. Next, a hierarchical Bayesian model performs a top-down adaptive circle fitting, with the ability to achieve a maximum a posteriori probability to fit circles to local image features. The evaluation of our approach with manufacturing images has demonstrated to be efficient in detecting small circles in machine parts.

Automatic welding of complex joints relies on scanning and inspection of the actual joint geometry, which is important for welding path planning and welding parameter setting. A robotic vision inspection system is presented in this paper to assess the geometry of complex joints before welding. The system can scan the joint with high accuracy and high resolutions via accurate system calibration and fast data synchronization. The scanned joint geometry is inspected to assess the quality of critical dimensions. Critical features are extracted for path planning, and the joint surface is segmented into different regions for assigning different welding parameters. Experimental results based on straight-joint and Y-joint are presented to show how the proposed system works.

Visual inspection involves dimensional inspection and surface inspection. Dimensional inspection is usually achieved via comparing the scanned model of the inspection target with the CAD model to detect dimensional defects such as bending and missing material. Surface inspection is usually based on textures via image processing and analysis to detect defects such as pitting, crack, scratch, and corrosion. Existing systems used fixed predefined viewpoints to capture geometry and texture from the object. However, defects may appear at different locations, distinguishable defect features may not be captured by the fixed viewpoints. Thus, dynamical planning of the viewpoints is desired to provide better sensing parameters adaptive to actual local geometry, appearance and lighting conditions. This paper proposes a new robotic sensing framework for adaptive visual inspection. Viewpoints are planned based on individual object adaptive to different defect locations. Sensor parameters at each viewpoints are adaptive to the surface conditions for optimized geometry/image quality. Data from all viewpoints are fused together to provide an adaptive model.

In this paper, we proposed an automatic crack detection method based on deep convolutional neural network -U-Net [4]. Unlike existing machine learning based crack detection methods, we can process an image as a whole without patchifying, thanks to the encoder-decoder structure of U-Net. The segmentation result is output from the network as a whole, instead of aggregation from neighborhood patches. In addition, a new cost function based on distance transform is introduced to assign pixel-level weight according to the minimal distance to the ground truth segmentation. In experiments, we test the proposed method on two datasets of road crack images. The pixel-level segmentation accuracy is above 92% which outperforms other state-of-the-art methods significantly.

In a 3-D scanning task, a robot-sensor system controls a robotic arm to move a laser sensor. In order to align the coordinate system of the robotic arm and laser sensor, prior calibration is required to derive the transformation between both coordinate systems. This paper proposes a new calibration method in three steps: manual data collection, sensing data calculation, and transformation solution. First, at least four data are required to be collected by the user. The sensing data are then calculated from the collected data and adopted to provide the desired transformation. The proposed algorithm has two features: arbitrary placement of planar disk and noise tolerant. Using a planar disk, the algorithm will automatically derive the angular relationship between the disk and the sensor plane, enabling arbitrary orientation placement. Noise tolerant is guaranteed by fitting ellipses during the sensing data calculation and using a single set of sensing data in transformation solution. Experiments and comparisons are given to demonstrate the efficiency of the proposed calibration algorithm.Note to Practitioners—This paper was motivated by the problem of calibrating a laser sensor and a positioning device (robot arm, CMM, etc.) in a robust and fast manner. Specifically, the calibration is to derive the transformation by aligning the sensors coordinate system to the positioning devices coordinate system. The proposed calibration procedure consists of two parts: manual data collection and automatic transformation calculation. During manual data collection, users only need to select four different data; whereby each data contains of two positions with the same orientation. Then, the desired transformation will be derived automatically. The calibration is designed in an efficient and robust way whereby: 1) data collection is done using a simple planar disk placed in arbitrary orientations; 2) minimum human interaction required; 3) tolerant to noise in the sensor data; and 4) easy implementation by following a proven and standard protocol.

The interactive image segmentation model allows users to iteratively add new inputs for refinement until a satisfactory result is finally obtained. Therefore, an ideal interactive segmentation model should learn to capture the user's intention with minimal interaction. However, existing models fail to fully utilize the valuable user input information in the segmentation refinement process and thus offer an unsatisfactory user experience. In order to fully exploit the user-provided information, we propose a new deep framework, called Regional Interactive Segmentation Network (RIS-Net), to expand the field-of-view of the given inputs to capture the local regional information surrounding them for local refinement. Additionally, RIS-Net adopts multiscale global contextual information to augment each local region for improving feature representation. We also introduce click discount factors to develop a novel optimization strategy for more effective end-to-end training. Comprehensive evaluations on four challenging datasets well demonstrate the superiority of the proposed RIS-Net over other state-of-the-art approaches.

This paper concerns multiphase piecewise smooth image segmentation with intensity inhomogeneities. Traditional methods based on the Mumford-Shah (MS) model require solving complicated diffusion equations evolving in irregular subdomains, leading to significant difficulties in efficient and accurate segmentation, especially in multiphase scenarios. In this paper, we propose a general framework to modify the MS model by using smoothing operators that can avoid the complicated implementation and inaccurate segmentation of traditional approaches. A detailed analysis connecting the smoothing operators and the diffusion equations is given to justify the modification. In addition, we present an efficient algorithm based on the direct augmented Lagrangian method, which requires fewer parameters than the commonly used augmented Lagrangian method. Typically, the smoothing operator in the general model is chosen to be Gaussian kernel, the bilateral kernel, and the directional diffusion kernel, respectively. Ample numerical results are provided to demonstrate the efficiency and accuracy of the modified model and the proposed minimization algorithm through various comparisons with existing approaches.

Objective evaluation of disease is one of the desirable goals in medicine. This paper presents a technique for the objective evaluation of facial paralysis, in which features are extracted based on landmark's positions in three-dimensional space (3D-landmarks). The landmarks are initialized manually in the first frontal frame and are tracked in the subsequent frontal frames. Then, the landmark's positions are reconstructed in 3D-space using multiview images and a camera self-calibration technique. From the 3D-landmarks, the features are extracted in 3D-space (called 3D-features) and used for classification. These 3D-features may contain enhanced information such as depth information and, therefore, may help improve the accuracy rates of predicted scores. In addition, our method uses the camera self-calibration technique for estimating the camera's parameters, and does not use laser scanning for 3D reconstruction, so it is more flexible to set up and safer for the patient. For overall evaluation, experiments showed that our technique achieved superior results to other methods.

The diagnosis of disease with the aid of computer programs has been developing more and more in recent years. This paper presents an approach which is based on frequency technique for the objective quantitative analysis of facial paralysis. In this method, limited-orientation modified circular Gabor filters (LO-MCGFs) are used to enhance the desirable frequencies in images. Then, features are extracted from the filtered images for classification. The first advantage of the LO-MCGF is that its inner passbands are uniform, so it helps remove noise and control frequencies more effectively. The second benefit is that the LO-MCGF utilizes the existing robust characteristics of circular Gabor filter for rotation invariant texture regions. Hence, the LO-MCGF-based technique improves remarkably the accuracies of score estimation for some expressions whose local textures are invariant in rotation. Finally, the limited filtered regions, or limited propagation orientations, help the LO-MCGF focus on only some specific spaces. Therefore, the LO-MCGF can avoid the influences of irrelevant regions. In other words, it improves the spatial localization. For overall evaluation, experiments show that our proposed method is superior to other contemporary techniques tested on a dynamic facial expression database.

We propose a cascaded and a contrario hole crack detection (CHORD) method for defect inspection in digital images of turbine blade surfaces. This is the first time an automatic image processing-based method is proposed for such a task. It consists of two major steps: first, the appearance of holes is approximated by ellipses and cascaded pose regression is used to estimate the position and orientation of the holes; Second, we define hole cracks as geometrical structures and a contrario method is used to assess the meaningfulness of each crack. The model-based CHORD method fully considers the features of hole cracks including the characteristics of brightness, length, and orientation, and therefore can accurately detect cracks in the images. The threshold on the strength of cracks is determined automatically and the computational time is about five seconds for each image.

Laser stripe extraction is essential to 3D reconstructions of objects by triangulation laser scanning. Industrial applications require robustness against the spurious points due to multiple reflections among reflective and uneven surfaces. Our approach is a three-stage algorithm, which first detects the potential laser stripe regions, then ranks the regions on their corrected scene irradiance densities, and finally conducts the column-wise peak detection using the region ranking. The proposed method is evaluated in terms of speed, accuracy and robustness on both simulations and a real application for steel groove inspection.

Variational image segmentation based on the Mumford-Shah model requires to solve the heat diffusion equations in evolving irregular subdomains. It brings about significant difficulties in efficient and accurate segmentation, especially, in multi-phase scenarios. In this paper, we propose a new Mumford-Shah type model involving a smoothing operator, acting a similar role as the diffusion process and avoiding complicated computation. We introduce two simplified numerical models based on the Gaussian operator and the bilateral operator respectively, and provide efficient numerical algorithms to solve both segmentation problems. Numerical validation experiments on various images show better accuracy and robustness over existing methods.

Image segmentation is to separate the image domain into sub-domains according to some optimization rule. Variational level-set image segmentation based on the Mumford-Shah model, together with level set formulation, leads to complicate implementation and expensive computational time. To overcome the difficulty in solving the complex coupled level set functions and handle any number of phases in the image, we improve the region-base active contour model in this paper. We formulate the image segmentation as a new multiphase minimization problem to handle images containing any integer phases of intensities. We also propose efficient numerical algorithms to solve such optimization problem without extra steps of convex relaxation. Numerical validation experiments on various images show better accuracy and robustness over existing methods.

Image segmentation is to separate the image domain into local regions. Implicit active contour models via the level-set method can provide smooth and closed contours. The major category is based on the Mumford-Shah image component decomposition model. Approximating the image domain by a set of homogeneous regions, segmentation based on the piecewise constant (PC) models fail in intensity inhomogeneous images. Requiring intensity smoothness in local regions, segmentation based on the piecewise smooth (PS) models can perform better for intensity inhomogeneity than those PC based. Existing PS-based techniques are inefficient and not robust to complicate intensity scenarios with noise. Here we introduce a new functional model by decomposing an image into three parts: PS components, PC components and noise components. We convexify the decomposition model by incorporating with relaxation techniques and optimize the PS components over the whole image domain. New numerical algorithms are also proposed to implement the above approaches efficiently. Numerical validation experiments show that the proposed approaches can achieve much faster, more robust and more accurate image segmentation than existing arts.

Neural stem cells and neural progenitors are early nervous system cells that form neurospheres when propagated in vitro. We study changes in growth using brightfield images to understand the effects of drugs. The image quality is generally poor, imposing challenges for automatic analysis. Level-set segmentation methods are able to handle topology changes but require close initializations for accurate and efficient results. Global level-set methods using single image-wide optimization objective functions are difficult to cope with large illumination and shading changes. We propose to adopt Hough transform to initialize localized level-sets for cell segmentation. Experimental results on 480 images with 738 neurospheres show that our proposed method performs best over existing level-set methods without appropriate initial contours.

In computational anatomy, statistical shape model (SSM) is used for the quantitative evaluation of variations in the shapes of different organs. This paper focuses on the construction of a SSM of the liver and its application to computer-assisted diagnosis of cirrhosis. We prove the potential application of SSMs in the classification of normal and cirrhotic livers. In constructing a SSM of the liver, we first normalize volume data followed by the construction of the model using principal component analysis. The coefficients of the model are used as indicators of liver pathology. The effectiveness of the constructed model is evaluated by the classification accuracy of both normal and abnormal data.

Automatic tumor detection and segmentation is essential for the computer-aided diagnosis of live tumors in CT images. However, it is a challenging task in low-contrast images as the low-level images are too weak to detect. In this paper, we propose a new method for the automatic detection of liver tumors. We first adaptively enhance the intensity contrast of CT images by probability density function estimation. Then, to detect tumorous regions, we use the expectation maximization/maximization of the posterior marginal (EM/MPM) algorithm, which utilizes both the intensity and label information of the adjacent regions. Finally, a shape constraint is applied to reduce noise and identify focal tumors. Quantitative evaluation experiments show that our method can accurately and effectively detect tumors even in poor-contrast CT images.

Automated clump decomposition is crucial for the analysis of neuronal cell fluorescent microscopic images due to inevitable nuclei and cell clumping. Existing techniques are not satisfactory in terms of accuracy and efficiency. In the current work, we propose a new method to automatically decompose nuclei clumps, using multiple visual cues from seeds, skeletons, and object boundaries and nuclei shape models to deliver robust and efficient analysis. The use of shape models enhances its robustness in handling complicated clumps against those bottom-up approaches without prior knowledge. We adopt a strategy of model verification in allowable local shape changes to improve the computational efficiency over template matching. Validation experiments in the analysis of 50 images containing over 2000 nuclei have demonstrated accuracy improvements over existing techniques.

Blastocystis is a unicellular but polymorphic protozoan parasite causing digestive diseases in humans. Autophagy, a self-degradation process, is only recently found in Blastocystis. Identifying and enumerating autophagic Blastocystis cells using fluorescent microscopy are important in biology. Doing this manually is laborious and error-prone. This paper proposes image analysis techniques to automate the process. The difficulties are poor image quality and large variations in illumination and cell morphology. We divide the cells into several sub-classes of different morphology. Support vector machines are used to learn domain knowledge and classify the cells. Validation experiments on separate data sets show reliable performance for manually segmented cells with sensitivity 82.2% and specificity 86.7%. For automatically segmented cells, the sensitivity is the same. However, the specificity drops down to 68.4%. To our knowledge, this is the first attempt in automatic processing these images.

Computer generation of cell images can provide annotated data to simulate various imaging conditions with controllable parameters. Synthesized images based on simple models cannot reflect the complicated parameter constraints in simulating real objects in terms of their deformation with appropriate probabilities. Learning-based techniques can provide insight to these properties and impose constraints on deformation selections. In this work, we discuss the simulation of gray level images of healthy red blood cell populations. Different from existing techniques, we learn the unbiased average shape and deformation models of the cells. Both models are used to guide the selection of possible deformations. We also learn cell color models to govern the texture generation of simulated cells. We apply this technique to simulate cell populations and validate the results using cell segmentation and counting algorithms. The proposed learning and simulation technique is generic and can be applied to other types of cells as well.

Decomposition of cells in clumps is a difficult segmentation task requiring region splitting techniques. Techniques that do not employ prior shape constraints usually fail to achieve accurate segmentation. Those using shape constraints are unable to cope with large clumps and occlusions. In this work, we propose a model-constrained region splitting algorithm for cell clump decomposition. We build the cell model using joint probability distribution of invariant shape features. The shape model, the contour smoothness and the gradient information along the cut are used to optimize the splitting in a recursive manner. The short cut rule is also adopted as a strategy to speed up the process. The algorithm performs well in validation experiments using 60 images with 4516 cells and 520 clumps.

The construction of probabilistic liver atlases has received little attention in the past. Existing methods are based on landmarks and are sensitive to their choices and placements. We propose an iterative landmark-free method based on dense volumes to construct linear unbiased diffeomorphic probabilistic atlases from liver CT images. The linear averaging of the transformed images is set as the common target space followed by pairwise diffeomorphic registrations to warp all images to the target using a recent-proposed efficient deformation approach during each iteration cycle. Iterative pairwise registrations are directly used to handle possible large deformations without the need for an extra step to remove global deformations such as the use of affine transformations in traditional methods. Compared with those approaches estimating the unbiased atlas and the transformations groupwise simultaneously, the current method is more efficient. The efficiency and the convergence of our method have been demonstrated experimentally by validation using 25 CT liver sets.

Automatic classification of working areas in peripheral blood smears can provide objective and reproducible quality control for the evaluation of smears and smear maker devices. However, it has drawn little research attention. In this paper we study this topic using image analysis and statistical pattern recognition methods. We employ generic features without requiring the extraction of individual cells. Two new spatial distribution features across scales are defined and utilized to classify working areas. We demonstrate that the only feature and method proposed in a similar work by others is insufficient to characterize the goodness of working areas, particularly the cell distribution. However, by utilizing it together with the features developed in this paper, we can achieve much better results. Our method has been tested on about 150 labeled images acquired from three malaria-infected Giemsa-stained blood smears using an oil immersion 100x objective lens.

Widely used in the evaluation of retrieval systems, the pooling method collects top ranked images from submitted retrieval systems resulting in possibly a very large pool of images. Inevitably, the pool may contain outliers. Human experts then manually annotate the relevance of them to create a ground truth for evaluation. Studies show that this annotation is time-consuming, tedious and inconsistent. To reduce human workload, this paper introduces an automatic method to detect outliers. Different from traditional detection methods using unsupervised techniques only, we utilize both supervised and unsupervised techniques sequentially as both positive and negative examples are (partially) available in this context. Specifically, support vector machines (SVMs) and fuzzy c-means clustering are used to predict data relevance and "outlierness". Performance improvements using our method after outlier removal have been validated on the medical image retrieval task in ImageCLEF 2004.

The paper addresses the problem of deformable hand shape recognition in biometric systems without positioning aids. We separate and recognize multiple rigid fingers. An elliptical model is introduced to represent fingers and accelerate the matching of them. Technically, our method bridges the traditional handcrafted-feature methods and the shape-distance methods. We have tested it using our 108-person, 540-sample database with significantly increased positive recognition accuracy.

This paper proposes a content-based medical image retrieval (CBMIR) framework using dynamically optimized features from multiple regions of medical images. These regional features, including structural and statistical properties of color, texture and geometry, are extracted from multiple dominant regions segmented by applying Gaussian mixture modeling (GMM) and the expectation maximization (EM) algorithm to medical images. Over them, principal component analysis (PCA) is utilized to construct query templates and to reduce feature dimensions for representative feature optimization. Applying this method to the tasks of the medical imageCLEF 2004 we achieve better retrieval performance (MAP 0.4535) over the existing work on casImage of about 9000 images.

We propose a novel framework for content-based image retrieval with multiple parallel retrieval engines (MultiPRE) to achieve higher retrieval performance. Visual features, including both low-level features, such as color, texture and region features, and middle-level structure features, such as blob representation of objects are used to capture geometrical and statistical characteristics of images. Both clustering analysis and discrimination analysis are used as similarity measures in multiple retrieval engines, which are based on~principal component analysis (PCA) and support vector machines (SVM), respectively. Finally outputs of these engines are fused to determine ranking lists of retrieved images for given retrieval topics. The proposed framework has been evaluated based on the 26 image query topics over the CasImage database~with over 9000 medical images~used in ImageCLEF 2004, an international research effort for content-based image retrieval performance benchmark. Experiments show that the proposed framework achieved significantly better performance in terms of both the mean and the variance of average precision than the best run reported in ImageCLEF2004.