The alignment of tissue between histopathological whole-slide-images (WSI) is crucial for research and clinical applications. Advances in computing, deep learning, and availability of large WSI datasets have revolutionised WSI analysis. Therefore, the current state-of-the-art in WSI registration is unclear. To address this, we conducted the ACROBAT challenge, based on the largest WSI registration dataset to date, including 4,212 WSIs from 1,152 breast cancer patients. The challenge objective was to align WSIs of tissue that was stained with routine diagnostic immunohistochemistry to its H&E-stained counterpart. We compare the performance of eight WSI registration algorithms, including an investigation of the impact of different WSI properties and clinical covariates. We find that conceptually distinct WSI registration methods can lead to highly accurate registration performances and identify covariates that impact performances across methods. These results provide a comparison of the performance of current WSI registration methods and guide researchers in selecting and developing methods.

Cross-modality image synthesis is an active research topic with multiple medical clinically relevant applications. Recently, methods allowing training with paired but misaligned data have started to emerge. However, no robust and well-performing methods applicable to a wide range of real world data sets exist. In this work, we propose a generic solution to the problem of cross-modality image synthesis with paired but non-aligned data by introducing new deformation equivariance encouraging loss functions. The method consists of joint training of an image synthesis network together with separate registration networks and allows adversarial training conditioned on the input even with misaligned data. The work lowers the bar for new clinical applications by allowing effortless training of cross-modality image synthesis networks for more difficult data sets.

SUMMARY: Digital pathology enables applying computational methods, such as deep learning, in pathology for improved diagnostics and prognostics, but lack of interoperability between whole slide image formats of different scanner vendors is a challenge for algorithm developers. We present OpenPhi-Open PatHology Interface, an Application Programming Interface for seamless access to the iSyntax format used by the Philips Ultra Fast Scanner, the first digital pathology scanner approved by the United States Food and Drug Administration. OpenPhi is extensible and easily interfaced with existing vendor-neutral applications. AVAILABILITY AND IMPLEMENTATION: OpenPhi is implemented in Python and is available as open-source under the MIT license at: https://gitlab.com/BioimageInformaticsGroup/openphi. The Philips Software Development Kit is required and available at: https://www.openpathology.philips.com. OpenPhi version 1.1.1 is additionally provided as Supplementary Data. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Nucleus detection is a fundamental task in histological image analysis and an important tool for many follow up analyses. It is known that sample preparation and scanning procedure of histological slides introduce a great amount of variability to the histological images and poses challenges for automated nucleus detection. Here, we studied the effect of histopathological sample fixation on the accuracy of a deep learning based nuclei detection model trained with hematoxylin and eosin stained images. We experimented with training data that includes three methods of fixation; PAXgene, formalin and frozen, and studied the detection accuracy results of various convolutional neural networks. Our results indicate that the variability introduced during sample preparation affects the generalization of a model and should be considered when building accurate and robust nuclei detection algorithms. Our dataset includes over 67 000 annotated nuclei locations from 16 patients and three different sample fixation types. The dataset provides excellent basis for building an accurate and robust nuclei detection model, and combined with unsupervised domain adaptation, the workflow allows generalization to images from unseen domains, including different tissues and images from different labs.

Automatic Non-rigid Histological Image Registration (ANHIR) challenge was organized to compare the performance of image registration algorithms on several kinds of microscopy histology images in a fair and independent manner. We have assembled 8 datasets, containing 355 images with 18 different stains, resulting in 481 image pairs to be registered. Registration accuracy was evaluated using manually placed landmarks. In total, 256 teams registered for the challenge, 10 submitted the results, and 6 participated in the workshop. Here, we present the results of 7 well-performing methods from the challenge together with 6 well-known existing methods. The best methods used coarse but robust initial alignment, followed by non-rigid registration, used multiresolution, and were carefully tuned for the data at hand. They outperformed off-the-shelf methods, mostly by being more robust. The best methods could successfully register over 98% of all landmarks and their mean landmark registration accuracy (TRE) was 0.44% of the image diagonal. The challenge remains open to submissions and all images are available for download.

Immunohistochemistry (IHC) of ER, PR, and Ki-67 are routinely used assays in breast cancer diagnostics. Determination of the proportion of stained cells (labeling index) should be restricted on malignant epithelial cells, carefully avoiding tumor infiltrating stroma and inflammatory cells. Here, we developed a deep learning based digital mask for automated epithelial cell detection using fluoro-chromogenic cytokeratin-Ki-67 double staining and sequential hematoxylin-IHC staining as training material. A partially pre-trained deep convolutional neural network was fine-tuned using image batches from 152 patient samples of invasive breast tumors. Validity of the trained digital epithelial cell masks was studied with 366 images captured from 98 unseen samples, by comparing the epithelial cell masks to cytokeratin images and by visual evaluation of the brightfield images performed by two pathologists. A good discrimination of epithelial cells was achieved (AUC of mean ROC = 0.93; defined as the area under mean receiver operating characteristics), and well in concordance with pathologists' visual assessment (4.01/5 and 4.67/5). The effect of epithelial cell masking on the Ki-67 labeling index was substantial. 52 tumor images initially classified as low proliferation (Ki-67 < 14%) without epithelial cell masking were re-classified as high proliferation (Ki-67 ≥ 14%) after applying the deep learning based epithelial cell mask. The digital epithelial cell masks were found applicable also to IHC of ER and PR. We conclude that deep learning can be applied to detect carcinoma cells in breast cancer samples stained with conventional brightfield IHC.

Cell counting from cell cultures is required in multiple biological and biomedical research applications. Especially, accurate brightfield-based cell counting methods are needed for cell growth analysis. With deep learning, cells can be detected with high accuracy, but manually annotated training data is required. We propose a method for cell detection that requires annotated training data for one cell line only, and generalizes to other, unseen cell lines. Training a deep learning model with one cell line only can provide accurate detections for similar unseen cell lines (domains). However, if the new domain is very dissimilar from training domain, high precision but lower recall is achieved. Generalization capabilities of the model can be improved with training data transformations, but only to a certain degree. To further improve the detection accuracy of unseen domains, we propose iterative unsupervised domain adaptation method. Predictions of unseen cell lines with high precision enable automatic generation of training data, which is used to train the model together with parts of the previously used annotated training data. We used U-Net-based model, and three consecutive focal planes from brightfield image z-stacks. We trained the model initially with PC-3 cell line, and used LNCaP, BT-474 and 22Rv1 cell lines as target domains for domain adaptation. Highest improvement in accuracy was achieved for 22Rv1 cells. F 1 -score after supervised training was only 0.65, but after unsupervised domain adaptation we achieved a score of 0.84. Mean accuracy for target domains was 0.87, with mean improvement of 16 percent. With our method for generalized cell detection, we can train a model that accurately detects different cell lines from brightfield images. A new cell line can be introduced to the model without a single manual annotation, and after iterative domain adaptation the model is ready to detect these cells with high accuracy.

Motivation: Digital pathology enables new approaches that expand beyond storage, visualization or analysis of histological samples in digital format. One novel opportunity is 3D histology, where a three-dimensional reconstruction of the sample is formed computationally based on serial tissue sections. This allows examining tissue architecture in 3D, for example, for diagnostic purposes. Importantly, 3D histology enables joint mapping of cellular morphology with spatially resolved omics data in the true 3D context of the tissue at microscopic resolution. Several algorithms have been proposed for the reconstruction task, but a quantitative comparison of their accuracy is lacking. Results: We developed a benchmarking framework to evaluate the accuracy of several free and commercial 3D reconstruction methods using two whole slide image datasets. The results provide a solid basis for further development and application of 3D histology algorithms and indicate that methods capable of compensating for local tissue deformation are superior to simpler approaches. Availability and implementation: Code: https://github.com/BioimageInformaticsTampere/RegBenchmark. Whole slide image datasets: http://urn.fi/urn: nbn: fi: csc-kata20170705131652639702. Supplementary information: Supplementary data are available at Bioinformatics online.

Studying tissue structure in 3D is beneficial in many applications. Reconstructing the structure based on histological sections has the advantages of high resolution and compatibility with conventional staining and interpretation techniques. However, obtaining an accurate 3D reconstruction based on a sequence of 2D sections is a difficult task. Evaluating the accuracy of such reconstructions is also challenging and it is often performed based only on visual inspections or a single indirect numerical measure. Here, we present a benchmarking framework composed of a panel of complementary metrics for assessing the quality of 3D reconstructions. We then apply the framework to evaluate the performance of several popular image registration algorithms in this context.

This paper proposes a framework for cell detection from bright-field microscope images. The method is trained using manually annotated images, and it uses Support Vector Machine classifiers with Histogram of Oriented Gradient features. The performance of the method is evaluated using 16 training and 12 test images with altogether 10736 human prostate cancer cells. Both the implementation and the annotated image database are released for download. The experiments consider various parameters and their effect on performance, and reaches accurate detection results with cross-validated AUC over 0.98, and mean relative deviation of 9 % from manually counted annotations in the growth curve over six days.

BACKGROUND: High-throughput genome-wide screening to study gene-specific functions, e.g. for drug discovery, demands fast automated image analysis methods to assist in unraveling the full potential of such studies. Image segmentation is typically at the forefront of such analysis as the performance of the subsequent steps, for example, cell classification, cell tracking etc., often relies on the results of segmentation. METHODS: We present a cell cytoplasm segmentation framework which first separates cell cytoplasm from image background using novel approach of image enhancement and coefficient of variation of multi-scale Gaussian scale-space representation. A novel outline-learning based classification method is developed using regularized logistic regression with embedded feature selection which classifies image pixels as outline/non-outline to give cytoplasm outlines. Refinement of the detected outlines to separate cells from each other is performed in a post-processing step where the nuclei segmentation is used as contextual information. RESULTS AND CONCLUSIONS: We evaluate the proposed segmentation methodology using two challenging test cases, presenting images with completely different characteristics, with cells of varying size, shape, texture and degrees of overlap. The feature selection and classification framework for outline detection produces very simple sparse models which use only a small subset of the large, generic feature set, that is, only 7 and 5 features for the two cases. Quantitative comparison of the results for the two test cases against state-of-the-art methods show that our methodology outperforms them with an increase of 4-9% in segmentation accuracy with maximum accuracy of 93%. Finally, the results obtained for diverse datasets demonstrate that our framework not only produces accurate segmentation but also generalizes well to different segmentation tasks.

In this paper, a novel method for retina verification based on minutiae features is proposed. The method uses the vessel direction information for improved matching robustness, and is thus suitable for cases where the overlapping region of the matched images is limited. Verification achieves a low false rejection ratio (FRR) even when the retinal area overlap is as low as 25% of the total area. The improved robustness is achieved using a new vessel-directional feature descriptor called the principal bifurcation orientation (PBO). The performance of the method is evaluated in terms of FRR and equal error rate (EER). In particular, the EER of 1.53% for the the difficult macula-papilla match with as little as 25% overlapping area is achieved.

MOTIVATION: Tissue heterogeneity, arising from multiple cell types, is a major confounding factor in experiments that focus on studying cell types, e.g. their expression profiles, in isolation. Although sample heterogeneity can be addressed by manual microdissection, prior to conducting experiments, computational treatment on heterogeneous measurements have become a reliable alternative to perform this microdissection in silico. Favoring computation over manual purification has its advantages, such as time consumption, measuring responses of multiple cell types simultaneously, keeping samples intact of external perturbations and unaltered yield of molecular content. RESULTS: We formalize a probabilistic model, DSection, and show with simulations as well as with real microarray data that DSection attains increased modeling accuracy in terms of (i) estimating cell-type proportions of heterogeneous tissue samples, (ii) estimating replication variance and (iii) identifying differential expression across cell types under various experimental conditions. As our reference we use the corresponding linear regression model, which mirrors the performance of the majority of current non-probabilistic modeling approaches. AVAILABILITY AND SOFTWARE: All codes are written in Matlab, and are freely available upon request as well as at the project web page http://www.cs.tut.fi/∼erkkila2/. Furthermore, a web-application for DSection exists at http://informatics.systemsbiology.net/DSection. CONTACT: [email protected]; [email protected]

BACKGROUND: Several algorithms have been proposed for detecting fluorescently labeled subcellular objects in microscope images. Many of these algorithms have been designed for specific tasks and validated with limited image data. But despite the potential of using extensive comparisons between algorithms to provide useful information to guide method selection and thus more accurate results, relatively few studies have been performed. RESULTS: To better understand algorithm performance under different conditions, we have carried out a comparative study including eleven spot detection or segmentation algorithms from various application fields. We used microscope images from well plate experiments with a human osteosarcoma cell line and frames from image stacks of yeast cells in different focal planes. These experimentally derived images permit a comparison of method performance in realistic situations where the number of objects varies within image set. We also used simulated microscope images in order to compare the methods and validate them against a ground truth reference result. Our study finds major differences in the performance of different algorithms, in terms of both object counts and segmentation accuracies. CONCLUSIONS: These results suggest that the selection of detection algorithms for image based screens should be done carefully and take into account different conditions, such as the possibility of acquiring empty images or images with very few spots. Our inclusion of methods that have not been used before in this context broadens the set of available detection methods and compares them against the current state-of-the-art methods for subcellular particle detection.

A computer vision system for automatic detection of interconnection points in inkjet printed interconnected electronics modules during the manufacturing process is described. The location data can be used for calculating the amount of misalignments and the required corrections. The method uses a local filtering operation for finding candidate regions, from which the false matches are filtered out by classification using an artificial neural network. Experiments show that atypical false match rate is as low as 0.18%. The location data is later used for correcting the wiring that is inkjetted on top of the layout to connect the connectors correctly.

Monitoring of bacterial populations requires automated analysis tools that provide accurate cell type quantification results. Here, methods for automated image analysis and bacteria type classification are presented. The classification method employs several discriminative features, calculated from automatically segmented images, for class determination. The performance of the algorithm is evaluated with a case study where three different bacterial types are present. Moreover, the accuracy of the method is demonstrated by generating experiments of synthetic bacterial population images.

Automated image analysis provides a powerful tool when quantifying various characteristics of cell populations. Previously, the validation of image analysis results has been a task of expert biologist, who has manually analyzed the images and provided the ground truth to which the proposed analysis results have been compared. The traditional validation approach, prone to errors and variation, is unfeasible in the emergence of high-throughput measurement systems which make human-based analysis excessively laborious. The systems biology approach for studying, e.g., cellular activity massively in parallel, often lending on high-throughput microscopy, further increases the need for efficient, validated computational methods. As a solution for the problem, we propose a computational framework for simulating fluorescence microscopy images of cell populations. The simulation framework allows generation of synthetic images with realistic characteristics including the ground truth for validation. Thus, the simulation enables validation and performance analysis for various analysis algorithms. By creating a parameterized model of cells based on a given population, the simulator is able to create different cell types. The proposed modular framework, combined with the ability to create high-throughput measurements, provides a powerful tool for validating image analysis methods in traditional microscopy as well as in high content screening. Moreover, we use experimental data to study the validity of the proposed modeling approach.

Fluorescence microscopy combined with digital imaging constructs a basic platform for numerous biomedical studies in the field of cellular imaging. As the studies relying on analysis of digital images have become popular, the validation of image processing methods used in automated image cytometry has become an important topic. Especially, the need for efficient validation has arisen from emerging high-throughput microscopy systems where manual validation is impractical. We present a simulation platform for generating synthetic images of fluorescence-stained cell populations with realistic properties. Moreover, we show that the synthetic images enable the validation of analysis methods for automated image cytometry and comparison of their performance. Finally, we suggest additional usage scenarios for the simulator. The presented simulation framework, with several user-controllable parameters, forms a versatile tool for many kinds of validation tasks, and is freely available at http://www.cs.tut.fi/sgn/csb/simcep.

MOTIVATION: Although numerous algorithms have been developed for microarray segmentation, extensive comparisons between the algorithms have acquired far less attention. In this study, we evaluate the performance of nine microarray segmentation algorithms. Using both simulated and real microarray experiments, we overcome the challenges in performance evaluation, arising from the lack of ground-truth information. The usage of simulated experiments allows us to analyze the segmentation accuracy on a single pixel level as is commonly done in traditional image processing studies. With real experiments, we indirectly measure the segmentation performance, identify significant differences between the algorithms, and study the characteristics of the resulting gene expression data. RESULTS: Overall, our results show clear differences between the algorithms. The results demonstrate how the segmentation performance depends on the image quality, which algorithms operate on significantly different performance levels, and how the selection of a segmentation algorithm affects the identification of differentially expressed genes. AVAILABILITY: Supplementary results and the microarray images used in this study are available at the companion web site http://www.cs.tut.fi/sgn/csb/spotseg/

BACKGROUND: Microarray technologies have become common tools in biological research. As a result, a need for effective computational methods for data analysis has emerged. Numerous different algorithms have been proposed for analyzing the data. However, an objective evaluation of the proposed algorithms is not possible due to the lack of biological ground truth information. To overcome this fundamental problem, the use of simulated microarray data for algorithm validation has been proposed. RESULTS: We present a microarray simulation model which can be used to validate different kinds of data analysis algorithms. The proposed model is unique in the sense that it includes all the steps that affect the quality of real microarray data. These steps include the simulation of biological ground truth data, applying biological and measurement technology specific error models, and finally simulating the microarray slide manufacturing and hybridization. After all these steps are taken into account, the simulated data has realistic biological and statistical characteristics. The applicability of the proposed model is demonstrated by several examples. CONCLUSION: The proposed microarray simulation model is modular and can be used in different kinds of applications. It includes several error models that have been proposed earlier and it can be used with different types of input data. The model can be used to simulate both spotted two-channel and oligonucleotide based single-channel microarrays. All this makes the model a valuable tool for example in validation of data analysis algorithms.

Perceptual image quality correlates ineffectively with the traditional error measures. By defining the factors behind perceived image quality, many image processing systems can be optimized. Thus, a great deal of effort has been made for solving this challenging issue. However, considerably less research has been focused on the role of spatial resolution and number of gray levels in subjective image quality. In our study, the relation between these two fundamental image parameters is experimentally defined in terms of perceptual quality. The study was carried out by organizing an experiment where 80 subjects were used. Using the collected data set, the results are illustrated in the form of isopreference curves. Finally, our results are compared with the corresponding ones presented previously. Although the comparison reveals a clear correlation between the results, novel information about the relation between spatial resolution and the number of gray levels is presented.