Magnetic Resonance Imaging (MRI) is a crucial modality for diagnosing knee joint diseases. However, accurately extracting disease-relevant features from complex multi-slice, multi-sequence MRI scans remains a considerable challenge. To address this, we propose 3D-HSPA, a novel diagnostic framework for multi-slice, multi-sequence knee MRI, which integrates disease-specific information at both slice and patch levels and establishes intrinsic spatial connections among different sequences. Specifically, we introduce a patch-level and slice-level label attention mechanism, guiding the model to automatically learn a precise alignment between image regions and disease labels. Furthermore, by mapping 2D images from various sequences into a unified 3D spatial coordinate system, we enhance the spatial consistency and robustness of the attention distributions. We validated 3D-HSPA on a large-scale MRI dataset comprising 50 fine-grained types of knee joint diseases. The experimental results demonstrate that 3D-HSPA not only achieves superior diagnostic performance but also exhibits strong model interpretability.

Magnetic Resonance Imaging (MRI) is crucial for diagnosing various knee-related diseases, and developing automatic diagnostic models based on knee MRI data is highly valuable. However, this task presents significant challenges due to the need to manage MRI data with multiple sequences and numerous images, where different diseases are often associated with specific images within certain sequences. To address these challenges, we propose a multi-label classification framework designed to effectively process MRI data and handle a large-scale label space encompassing hundreds of disease categories. Our approach introduces a Slice-Level Label Attention mechanism, which enables the model to learn the alignment between labels and images within sequences, thereby enhancing both performance and interpretability. Additionally, we present a Global-Guided Attention Regularization mechanism that further improves the consistency and robustness of the Slice-Level Label Attention results. We validate our framework on a large-scale MRI dataset involving multi-label classification across hundreds of fine-grained disease categories. Experimental results demonstrate that our method not only achieves superior performance but also provides more robust and consistent interpretability.

For surgical planning and intra-operation imaging, CT reconstruction using X-ray images can potentially be an important alternative when CT imaging is not available or not feasible. In this paper, we aim to use biplanar X-rays to reconstruct a 3D CT image, because biplanar X-rays convey richer information than single-view X-rays and are more commonly used by surgeons. Different from previous studies in which the two X-ray views were treated indifferently when fusing the cross-view data, we propose a novel attention-informed coarse-to-fine cross-view fusion method to combine the features extracted from the orthogonal biplanar views. This method consists of a view attention alignment sub-module and a fine-distillation sub-module that are designed to work together to highlight the unique or complementary information from each of the views. Experiments have demonstrated the superiority of our proposed method over the SOTA methods.

Medical imaging plays a pivotal role in clinical diagnosis, and the textual reports associated with these images are of paramount importance in aiding image comprehension and supporting treatment decisions. Automated report generation serves to alleviate the burden on radiologists and has garnered significant attention in the field of medical artificial intelligence. Previous research in text-based report generation primarily focused on generating impressions statements from radiology findings. However, the benefits in terms of reducing the workload on radiologists were not particularly evident. In this article, we propose a novel task of generating findings from radiology impressions. Leveraging advanced large language models, we trained a set of report generation models using a real dataset of knee MRI reports. Additionally, we incorporated various strategies, including data augmentation and efficient parameter fine-tuning. Objective experiments affirm the effectiveness of the methods we introduced. Furthermore, we conducted subjective assessments by radiologists, and the results demonstrate that our trained large language models significantly outperform professional radiologists in terms of overall report quality and content consistency.