With the rapid advancement of pathology foundation models (FMs), the representation learning of whole slide images (WSIs) attracts increasing attention. Existing studies develop high-quality patch feature extractors and employ carefully designed aggregation schemes to derive slide-level representations. However, mainstream weakly supervised slide representation learning methods, primarily based on multiple instance learning (MIL), are tailored to specific downstream tasks, which limits their generalizability. To address this issue, some studies explore unsupervised slide representation learning. However, these approaches focus solely on the visual modality of patches, neglecting the rich semantic information embedded in textual data. In this work, we propose ProAlign, a cross-modal unsupervised slide representation learning framework. Specifically, we leverage a large language model (LLM) to generate descriptive text for the prototype types present in a WSI, introducing patch-text contrast to construct initial prototype embeddings. Furthermore, we propose a parameter-free attention aggregation strategy that utilizes the similarity between patches and these prototypes to form unsupervised slide embeddings applicable to a wide range of downstream tasks. Extensive experiments on four public datasets show that ProAlign outperforms existing unsupervised frameworks and achieves performance comparable to some weakly supervised models.

Recent advancements in computational pathology and artificial intelligence have significantly improved whole slide image (WSI) classification. However, the gigapixel resolution of WSIs and the scarcity of manual annotations present substantial challenges. Multiple instance learning (MIL) is a promising weakly supervised learning approach for WSI classification. Recently research revealed employing pseudo bag augmentation can encourage models to learn various data, thus bolstering models' performance. While directly inheriting the parents' labels can introduce more noise by mislabeling in training. To address this issue, we translate the WSI classification task from weakly supervised learning to semi-weakly supervised learning, termed SWS-MIL, where adaptive pseudo bag augmentation (AdaPse) is employed to assign labeled and unlabeled data based on a threshold strategy. Using the “student-teacher” pattern, we introduce a feature augmentation technique, MergeUp, which merges bags with low-priority bags to enhance inter-category information, increasing training data diversity. Experimental results on the CAMELYON-16, BRACS, and TCGA-LUNG datasets demonstrate the superiority of our method over existing state-of-the-art approaches, affirming its efficacy in WSI classification.

Multiple Instance Learning (MIL) consists of two stages: feature encoding of instances and feature fusion of instances. This paper identifies two issues in the feature fusion stage of MIL when applied to pathological image classification, which creates performance bottlenecks. First, previous MIL methods lack positional encoding in the feature fusion stage. However, the number of positive instances is critical for pathological diagnosis. This makes it difficult for earlier MIL models to perceive positive instances, limiting their ability to capture the semantic correlation between the number of positive instances and the disease. Second, previous MIL methods base both inference and training on a single Whole Slide Image (WSI), failing to utilize cross-slide information effectively. To address these issues, this paper proposes a novel attention mechanism called CORE Attention (Contextual Position Encoding-based Retrievable Attention) during the instance fusion stage of MIL, and develops the COREMIL model based on it. CORE Attention consists of two modules: contextual position encoding and cross-slide retrieval-based attention fine-tuning. The contextual position encoding captures better contextual information, especially for counting tasks. Cross-slide retrieval-based attention fine-tuning allows the model to leverage previously learned historical information to guide the attention in the current feature fusion process. This paper validated COREMIL’s classification performance on several public datasets and a private dataset of pathological images. Extensive experiments demonstrate that COREMIL outperforms other current MIL models in terms of F1 Score and AUC on most datasets, with improvements of up to 14.414%. This CORE Attention-based approach offers an efficient solution for slide-level classification problems. Our code will be accessed shortly.

Layer pruning has emerged as a widely used technique for compressing large language models (LLMs). However, existing layer pruning approaches often incur substantial performance degradation. We identify the majority of this degradation to a single yet previously overlooked issue: \textit{the mismatch of activation magnitudes at the pruning interface}. The pre-interface activations exhibit significantly different scales from the post-interface ones, causing the distributional shift as it propagates through the remaining layers. To address this issue, we introduce \textsc{LinearPatch}, a lightweight and plug-and-play technique that fuses two operations into one matrix multiply at the pruning interface: (i) a Hadamard transformation that suppresses massive outliers at particular tokens and (ii) a channel-wise scaling that aligns activation statistics. On LLaMA-3-8B, \textsc{LinearPatch} preserves up to \textbf{94.15\%} of the original model's performance when pruning 5 out of 32 layers, outperforming the previous state of the art by \textbf{4\%}. The patch can be further refined with 5K unlabeled samples via memory-efficient offline distillation, pushing the retention to 95.16\% within only 30 minutes on a single GPU. Code is available at \url{https://github.com/chenxinrui-tsinghua/LinearPatch}.

Quantization has enabled the widespread implementation of deep learning algorithms on resource-constrained Internet of Things (IoT) devices, which compresses neural networks by reducing the bit-width of their parameters. However, most quantization methods invade privacy as they require real training datasets for calibration or fine-tuning. As a solution, zero-shot quantization (ZSQ) has emerged as a paradigm to quantize neural networks without accessing training datasets. Most employ data generation schemes to synthesize calibration data for knowledge transfer from the full-precision networks to the quantized ones. For privacy-protected and resource-constrained IoT devices, achieving optimal deployment necessitates the strategic integration of synthetic data generation and low-bit-width quantization techniques. However, when it comes to the lower bit-width case in ZSQ, we observe that the discrepancy between the full-precision network and the quantized network tends to widen significantly, hindering the knowledge transfer, which is attributed to the three following challenges: 1) hard logits matching with wide discrepancy; 2) unstable feature alignment with huge quantization error; and 3) synthetic data with low diversity. To address these issues, this article presents S-ZSQ, a novel ZSQ framework with two-pronged strategies that enhances both knowledge transfer and synthetic data generation, which enables low-bit-width quantized network to derive more soft feature-infused hints from the full-precision network. We achieve significant improvements on classification tasks, including CIFAR-10/100 and ImageNet-1k, with fewer fine-tuning epochs, particularly in scenarios involving low-bit-width quantization. For example, in the 3-bit ResNet-18/ResNet-50 case, we outperform AdaDFQ by 8.08%/11.16% in top-1 accuracy on ImageNet-1k.

Accurate diagnosis of cerebral tumors is crucial for effective clinical therapeutics and prognosis. However, limitations in brain biopsy tissues and the scarcity of pathologists specializing in cerebral tumors hinder comprehensive clinical tests for precise diagnosis. To address these challenges, we first established a brain tumor dataset of 3,520 cases collected from multiple centers. We then proposed a novel Hierarchically Optimized Multiple Instance Learning (HOMIL) method for classifying six common brain tumor types, glioma grading, and predicting the origin of brain metastatic cancers. The feature encoder and aggregator in HOMIL were trained alternately based on specific datasets and tasks. Compared to other multiple instance learning (MIL) methods, HOMIL achieved state-of-the-art performance with impressive accuracies: 93.29% / 85.60% for brain tumor classification, 91.21% / 96.93% for glioma grading, and 86.36% / 79.28% for origin determination on internal/external datasets. Additionally, HOMIL effectively located multi-scale regions of interest, enabling an in-depth analysis through features and heatmaps. Extensive visualization demonstrated HOMIL's ability to cluster features within the same type while establishing distinct boundaries between tumor types. It also identified critical areas on pathological slides, regardless of tumor size.

In computational pathology, whole-slide image (WSI) classification presents a formidable challenge due to its gigapixel resolution and limited fine-grained annotations. Multiple-instance learning (MIL) offers a weakly supervised solution, yet refining instance-level information from bag-level labels remains challenging. While most of the conventional MIL methods use attention scores to estimate instance importance scores (IIS) which contribute to the prediction of the slide labels, these often lead to skewed attention distributions and inaccuracies in identifying crucial instances. To address these issues, we propose a new approach inspired by cooperative game theory: employing Shapley values to assess each instance's contribution, thereby improving IIS estimation. The computation of the Shapley value is then accelerated using attention, meanwhile retaining the enhanced instance identification and prioritization. We further introduce a framework for the progressive assignment of pseudo bags based on estimated IIS, encouraging more balanced attention distributions in MIL models. Our extensive experiments on CAMELYON-16, BRACS, TCGA-LUNG, and TCGA-BRCA datasets show our method's superiority over existing state-of-the-art approaches, offering enhanced interpretability and class-wise insights. Our source code is available at https://github.com/RenaoYan/PMIL.

The two primary types of Hematoxylin and Eosin (H&E) slides in histopathology are Formalin-Fixed Paraffin-Embedded (FFPE) and Fresh Frozen (FF). FFPE slides offer high quality histopathological images but require a labor-intensive acquisition process. In contrast, FF slides can be prepared quickly, but the image quality is relatively poor. Our task is to translate FF images into FFPE style, thereby improving the image quality for diagnostic purposes. In this paper, we propose Diffusion-FFPE, a method for FF-to-FFPE histopathological image translation using a pre-trained diffusion model. Specifically, we utilize a one-step diffusion model as the generator, which we fine-tune using LoRA adapters within an adversarial learning framework. To enable the model to effectively capture both global structural patterns and local details, we introduce a multi-scale feature fusion module that leverages two VAE encoders to extract features at different image resolutions, performing feature fusion before inputting them into the UNet. Additionally, a pre-trained vision-language model for histopathology serves as the backbone for the discriminator, enhancing model performance. Our FF-to-FFPE translation experiments on the TCGA-NSCLC dataset demonstrate that the proposed approach outperforms existing methods. The code and models are released at https://github.com/QilaiZhang/Diffusion-FFPE.

Histopathological whole slide images (WSIs) classification has become a foundation task in medical microscopic imaging processing. Prevailing approaches involve learning WSIs as instance-bag representations, emphasizing significant instances but struggling to capture the interactions between instances. Additionally, conventional graph representation methods utilize explicit spatial positions to construct topological structures but restrict the flexible interaction capabilities between instances at arbitrary locations, particularly when spatially distant. In response, we propose a novel dynamic graph representation algorithm that conceptualizes WSIs as a form of the knowledge graph structure. Specifically, we dynamically construct neighbors and directed edge embeddings based on the head and tail relationships between instances. Then, we devise a knowledge-aware attention mechanism that can update the head node features by learning the joint attention score of each neighbor and edge. Finally, we obtain a graph-level embedding through the global pooling process of the updated head, serving as an implicit representation for the WSI classification. Our end-to-end graph representation learning approach has outperformed the state-of-the-art WSI analysis methods on three TCGA benchmark datasets and in-house test sets. Our code is available at https://github.com/WonderLandxD/WiKG.

To deploy neural networks on clinical edge devices, quantization is the most commonly used method to compress the models, which requires a calibration set of hundreds of real images. However, due to privacy concerns, the scarcity of private histopathological images hinders the application of quantization. To address this issue, we develop HIQ, a novel one-shot quantization framework for histopathological image classification networks, which requires only one real image per class for calibration. To compensate for data scarcity, sample BNS alignment is introduced to generate synthetic images with similar distribution to the real ones. To improve the diversity of synthetic images, fine-grained diversity enhancement that provides fine-grained enhancement intensity for different classes and network layers is proposed, based on the observation of the class-wise and layer-wise fine-grained data. Finally, the asymptotic enhancement strategy is highlighted to achieve a trade-off between inter-class distance and intra-class diversity of synthetic images, based on the insight of the smaller inter-class distance of histopathological images than that of natural ones. Extensive experiments on the BRACS dataset show that our method achieves an extremely low accuracy loss even compared to the full precision model in low-bit cases and maintains robustness when missing classes of real images.

Histopathology analysis is the gold standard for medical diagnosis. Accurate classification of whole slide images (WSIs) and region-of-interests (ROIs) localization can assist pathologists in diagnosis. The gigapixel resolution of WSI and the absence of fine-grained annotations make direct classification and analysis challenging. In weakly supervised learning, multiple instance learning (MIL) presents a promising approach for WSI classification. The prevailing strategy is to use attention mechanisms to measure instance importance for classification. However, attention mechanisms fail to capture inter-instance information, and self-attention causes quadratic computational complexity. To address these challenges, we propose AMD-MIL, an agent aggregator with a mask denoise mechanism. The agent token acts as an intermediate variable between the query and key for computing instance importance. Mask and denoising matrices, mapped from agents-aggregated value, dynamically mask low-contribution representations and eliminate noise. AMD-MIL achieves better attention allocation by adjusting feature representations, capturing micro-metastases in cancer, and improving interpretability. Extensive experiments on CAMELYON-16, CAMELYON-17, TCGA-KIDNEY, and TCGA-LUNG show AMD-MIL's superiority over state-of-the-art methods.

Digitization of pathological slides has promoted the research of computer-aided diagnosis, in which artificial intelligence analysis of pathological images deserves attention. Appropriate deep learning techniques in natural images have been extended to computational pathology. Still, they seldom take into account prior knowledge in pathology, especially the analysis process of lesion morphology by pathologists. Inspired by the diagnosis decision of pathologists, we design a novel deep learning architecture based on tree-like strategies called DeepTree. It imitates pathological diagnosis methods, designed as a binary tree structure, to conditionally learn the correlation between tissue morphology, and optimizes branches to finetune the performance further. To validate and benchmark DeepTree, we build a dataset of frozen lung cancer tissues and design experiments on a public dataset of breast tumor subtypes and our dataset. Results show that the deep learning architecture based on tree-like strategies makes the pathological image classification more accurate, transparent, and convincing. Simultaneously, prior knowledge based on diagnostic strategies yields superior representation ability compared to alternative methods. Our proposed methodology helps improve the trust of pathologists in artificial intelligence analysis and promotes the practical clinical application of pathology-assisted diagnosis.

Pathological whole slide images (WSIs) are of great importance for clinical diagnosis, and the classification of WSIs is often regarded as a multiple instance learning (MIL) problem. However, most MIL frameworks only use bag labels to train their models, neglecting the potential information from negative bags, which is a waste of existing labeled information. In this paper, we propose a semantic-similarity collaborative knowledge distillation framework (SSC) for WSI classification, which differs from other methods by leveraging the latent information of the data to the fullest extent and considering both semantic space and similarity space, then transferring this knowledge to a lightweight network through knowledge distillation, achieving end-to-end fast inference. Specifically, our proposed method includes three branches: the teacher, student, and regular branches. The teacher branch explores the similarity space and includes a similarity-based classifier that outputs classification results based on the similarity space mapped from negative bags. The output of the similarity-based classifier is fused with that of the semantic-based classifier and serves as the soft pseudo-label for the student branch. To improve the quality of the labels, we use views under different augmentation modes as inputs for the three branches and encourage weakly augmented features to be as similar as possible to strongly augmented features through consistency regularization. Moreover, the teacher, student, and regular branches share the weights of the classifier and projection head, thus strengthening their knowledge exchange. The proposed method exhibits excellent performance on Camelyon16.

Quantization is an effective way to compress neural networks. By reducing the bit width of the parameters, the processing efficiency of neural network models at edge devices can be notably improved. Most conventional quantization methods utilize real datasets to optimize quantization parameters and fine-tune. Due to the inevitable privacy and security issues of real samples, the existing real-data-driven methods are no longer applicable. Thus, a natural method is to introduce synthetic samples for zero-shot quantization (ZSQ). However, the conventional synthetic samples fail to retain the detailed texture feature distributions, which severely limits the knowledge transfer and performance of the quantized model. In this paper, a novel ZSQ method, TexQ is proposed to address this issue. We first synthesize a calibration image and extract its calibration center for each class with a texture feature energy distribution calibration method. Then, the calibration centers are used to guide the generator to synthesize samples. Finally, we introduce the mixup knowledge distillation module to diversify synthetic samples for fine-tuning. Extensive experiments on CIFAR10/100 and ImageNet show that TexQ is observed to perform state-of-the-art in ultra-low bit width quantization. For example, when ResNet-18 is quantized to 3-bit, TexQ achieves a 12.18% top-1 accuracy increase on ImageNet compared to state-of-the-art methods. Code at https://github.com/dangsingrue/TexQ.

Environment perception is the premise for intelligent vehicles to drive safely and stably. Despite the rapid development of road detection technology based on visual images, it is still challenging to robustly identify road areas in visual images due to the influence of illumination changes and noise. In order to solve this problem, we introduce a new optimized lidar and camera sensor fusion method for road environment sensing of intelligent vehicles. In road boundary detection based on laser data, a median point filtering method of ordered pole cloud is proposed. A method of boundary search, boundary seed point growth and obstacle clustering is proposed to identify road boundary. In the lane line classification based on visual image, a lane line search classification method is proposed, which can effectively classify lane lines and extract single lane lines. On the basis of the optimization of sensors, several constraint conditions are proposed based on the fusion of the two data, and the location of missing lane lines is predicted by using the road information identified by lidar and image, and the lane lines are identified again. Finally, a large number of experiments are carried out on kitti-Road benchmark data set, and a test platform is built to verify the results of the identification method proposed in this paper in rainy day, cloudy day, night and other special scenarios. Experimental results show that this method is superior to existing methods.

Pathological examination is the gold standard for the diagnosis of cancer. Common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry (IHC). In some cases, it is hard to make accurate diagnoses of cancer by referring only to H&E staining images. Whereas, the IHC examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual IHC images from H&E-stained images will be a good solution for current IHC examination hard accessibility issue, especially for some low-resource regions. However, existing approaches have limitations in microscopic structural preservation and the consistency of pathology properties. In addition, pixel-level paired data is hard available. In our work, we propose a novel adversarial learning method for effective Ki-67-stained image generation from corresponding H&E-stained image. Our method takes fully advantage of structural similarity constraint and skip connection to improve structural details preservation; and pathology consistency constraint and pathological representation network are first proposed to enforce the generated and source images hold the same pathological properties in different staining domains. We empirically demonstrate the effectiveness of our approach on two different unpaired histopathological datasets. Extensive experiments indicate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin. In addition, our approach also achieves a stable and good performance on unbalanced datasets, which shows our method has strong robustness. We believe that our method has significant potential in clinical virtual staining and advance the progress of computer-aided multi-staining histology image analysis.

The paper studied relative motion equation for satellite formation flying with large separations. The configuration is traditionally designed by the periodic solutions of the C-W equation in circle reference orbit or Lawden equation in elliptic reference orbit. Hence, the linear solutions are more suitable for the configuration with small scale formation than large scale formation. However, in some specific situations, it is necessary to use satellites with large separations. Then the paper studied relative motion based on the nonlinear equations in an elliptic reference orbit. The solution is expanded as series form with respect to the eccentricity of the reference orbit, in-plane amplitude and out-of-plane amplitude. Taking the Lawden periodic solution as starting point, the high-order analytical solution is constructed by Lindstedt-Poincare method. Particularly, as the eccentricity is zero, the analytical solution degenerated to express the relative motion in circle reference orbit. Finally, the practical convergence of the analytical solution is discussed in order to examine its validity and applicability.