Practical deployment of Vision Transformers in computational pathology has largely been constrained by the sheer size of whole-slide images. Transformers faced a similar limitation when applied to long documents, and Hierarchical Transformers were introduced to circumvent it. This work explores the capabilities of Hierarchical Vision Transformers for prostate cancer grading in WSIs and presents a novel technique to combine attention scores smartly across hierarchical transformers. Our best-performing model matches state-of-the-art algorithms with a 0.916 quadratic kappa on the Prostate cANcer graDe Assessment (PANDA) test set. It exhibits superior generalization capacities when evaluated in more diverse clinical settings, achieving a quadratic kappa of 0.877, outperforming existing solutions. These results demonstrate our approach's robustness and practical applicability, paving the way for its broader adoption in computational pathology and possibly other medical imaging tasks. Our code is publicly available at https://github.com/computationalpathologygroup/hvit.

The alignment of tissue between histopathological whole-slide-images (WSI) is crucial for research and clinical applications. Advances in computing, deep learning, and availability of large WSI datasets have revolutionised WSI analysis. Therefore, the current state-of-the-art in WSI registration is unclear. To address this, we conducted the ACROBAT challenge, based on the largest WSI registration dataset to date, including 4,212 WSIs from 1,152 breast cancer patients. The challenge objective was to align WSIs of tissue that was stained with routine diagnostic immunohistochemistry to its H&E-stained counterpart. We compare the performance of eight WSI registration algorithms, including an investigation of the impact of different WSI properties and clinical covariates. We find that conceptually distinct WSI registration methods can lead to highly accurate registration performances and identify covariates that impact performances across methods. These results provide a comparison of the performance of current WSI registration methods and guide researchers in selecting and developing methods.

BACKGROUND: Stochastic optical reconstruction microscopy (STORM), a super-resolution microscopy technique based on single-molecule localizations, has become popular to characterize sub-diffraction limit targets. However, due to lengthy image acquisition, STORM recordings are prone to sample drift. Existing cross-correlation or fiducial marker-based algorithms allow correcting the drift within each channel, but misalignment between channels remains due to interchannel drift accumulating during sequential channel acquisition. This is a major drawback in multi-color STORM, a technique of utmost importance for the characterization of various biological interactions. RESULTS: We developed RegiSTORM, a software for reducing channel misalignment by accurately registering STORM channels utilizing fiducial markers in the sample. RegiSTORM identifies fiducials from the STORM localization data based on their non-blinking nature and uses them as landmarks for channel registration. We first demonstrated accurate registration on recordings of fiducials only, as evidenced by significantly reduced target registration error with all the tested channel combinations. Next, we validated the performance in a more practically relevant setup on cells multi-stained for tubulin. Finally, we showed that RegiSTORM successfully registers two-color STORM recordings of cargo-loaded lipid nanoparticles without fiducials, demonstrating the broader applicability of this software. CONCLUSIONS: The developed RegiSTORM software was demonstrated to be able to accurately register multiple STORM channels and is freely available as open-source (MIT license) at https://github.com/oystein676/RegiSTORM.git and https://doi.org/10.5281/zenodo.5509861 (archived), and runs as a standalone executable (Windows) or via Python (Mac OS, Linux).

MOTIVATION: Molecular phenotyping by gene expression profiling is central in contemporary cancer research and in molecular diagnostics but remains resource intense to implement. Changes in gene expression occurring in tumours cause morphological changes in tissue, which can be observed on the microscopic level. The relationship between morphological patterns and some of the molecular phenotypes can be exploited to predict molecular phenotypes from routine haematoxylin and eosin-stained whole slide images (WSIs) using convolutional neural networks (CNNs). In this study, we propose a new, computationally efficient approach to model relationships between morphology and gene expression. RESULTS: We conducted the first transcriptome-wide analysis in prostate cancer, using CNNs to predict bulk RNA-sequencing estimates from WSIs for 370 patients from the TCGA PRAD study. Out of 15 586 protein coding transcripts, 6618 had predicted expression significantly associated with RNA-seq estimates (FDR-adjusted P-value <1×10-4) in a cross-validation and 5419 (81.9%) of these associations were subsequently validated in a held-out test set. We furthermore predicted the prognostic cell-cycle progression score directly from WSIs. These findings suggest that contemporary computer vision models offer an inexpensive and scalable solution for prediction of gene expression phenotypes directly from WSIs, providing opportunity for cost-effective large-scale research studies and molecular diagnostics. AVAILABILITY AND IMPLEMENTATION: A self-contained example is available from http://github.com/phiwei/prostate_coexpression. Model predictions and metrics are available from doi.org/10.5281/zenodo.4739097. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

SUMMARY: Digital pathology enables applying computational methods, such as deep learning, in pathology for improved diagnostics and prognostics, but lack of interoperability between whole slide image formats of different scanner vendors is a challenge for algorithm developers. We present OpenPhi-Open PatHology Interface, an Application Programming Interface for seamless access to the iSyntax format used by the Philips Ultra Fast Scanner, the first digital pathology scanner approved by the United States Food and Drug Administration. OpenPhi is extensible and easily interfaced with existing vendor-neutral applications. AVAILABILITY AND IMPLEMENTATION: OpenPhi is implemented in Python and is available as open-source under the MIT license at: https://gitlab.com/BioimageInformaticsGroup/openphi. The Philips Software Development Kit is required and available at: https://www.openpathology.philips.com. OpenPhi version 1.1.1 is additionally provided as Supplementary Data. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Automatic Non-rigid Histological Image Registration (ANHIR) challenge was organized to compare the performance of image registration algorithms on several kinds of microscopy histology images in a fair and independent manner. We have assembled 8 datasets, containing 355 images with 18 different stains, resulting in 481 image pairs to be registered. Registration accuracy was evaluated using manually placed landmarks. In total, 256 teams registered for the challenge, 10 submitted the results, and 6 participated in the workshop. Here, we present the results of 7 well-performing methods from the challenge together with 6 well-known existing methods. The best methods used coarse but robust initial alignment, followed by non-rigid registration, used multiresolution, and were carefully tuned for the data at hand. They outperformed off-the-shelf methods, mostly by being more robust. The best methods could successfully register over 98% of all landmarks and their mean landmark registration accuracy (TRE) was 0.44% of the image diagonal. The challenge remains open to submissions and all images are available for download.

Motivation: Digital pathology enables new approaches that expand beyond storage, visualization or analysis of histological samples in digital format. One novel opportunity is 3D histology, where a three-dimensional reconstruction of the sample is formed computationally based on serial tissue sections. This allows examining tissue architecture in 3D, for example, for diagnostic purposes. Importantly, 3D histology enables joint mapping of cellular morphology with spatially resolved omics data in the true 3D context of the tissue at microscopic resolution. Several algorithms have been proposed for the reconstruction task, but a quantitative comparison of their accuracy is lacking. Results: We developed a benchmarking framework to evaluate the accuracy of several free and commercial 3D reconstruction methods using two whole slide image datasets. The results provide a solid basis for further development and application of 3D histology algorithms and indicate that methods capable of compensating for local tissue deformation are superior to simpler approaches. Availability and implementation: Code: https://github.com/BioimageInformaticsTampere/RegBenchmark. Whole slide image datasets: http://urn.fi/urn: nbn: fi: csc-kata20170705131652639702. Supplementary information: Supplementary data are available at Bioinformatics online.

Studying tissue structure in 3D is beneficial in many applications. Reconstructing the structure based on histological sections has the advantages of high resolution and compatibility with conventional staining and interpretation techniques. However, obtaining an accurate 3D reconstruction based on a sequence of 2D sections is a difficult task. Evaluating the accuracy of such reconstructions is also challenging and it is often performed based only on visual inspections or a single indirect numerical measure. Here, we present a benchmarking framework composed of a panel of complementary metrics for assessing the quality of 3D reconstructions. We then apply the framework to evaluate the performance of several popular image registration algorithms in this context.

BACKGROUND: Orientation and the degree of isotropy are important in many biological systems such as the sarcomeres of cardiomyocytes and other fibrillar structures of the cytoskeleton. Image based analysis of such structures is often limited to qualitative evaluation by human experts, hampering the throughput, repeatability and reliability of the analyses. Software tools are not readily available for this purpose and the existing methods typically rely at least partly on manual operation. RESULTS: We developed CytoSpectre, an automated tool based on spectral analysis, allowing the quantification of orientation and also size distributions of structures in microscopy images. CytoSpectre utilizes the Fourier transform to estimate the power spectrum of an image and based on the spectrum, computes parameter values describing, among others, the mean orientation, isotropy and size of target structures. The analysis can be further tuned to focus on targets of particular size at cellular or subcellular scales. The software can be operated via a graphical user interface without any programming expertise. We analyzed the performance of CytoSpectre by extensive simulations using artificial images, by benchmarking against FibrilTool and by comparisons with manual measurements performed for real images by a panel of human experts. The software was found to be tolerant against noise and blurring and superior to FibrilTool when analyzing realistic targets with degraded image quality. The analysis of real images indicated general good agreement between computational and manual results while also revealing notable expert-to-expert variation. Moreover, the experiment showed that CytoSpectre can handle images obtained of different cell types using different microscopy techniques. Finally, we studied the effect of mechanical stretching on cardiomyocytes to demonstrate the software in an actual experiment and observed changes in cellular orientation in response to stretching. CONCLUSIONS: CytoSpectre, a versatile, easy-to-use software tool for spectral analysis of microscopy images was developed. The tool is compatible with most 2D images and can be used to analyze targets at different scales. We expect the tool to be useful in diverse applications dealing with structures whose orientation and size distributions are of interest. While designed for the biological field, the software could also be useful in non-biological applications.