The basal diameter of uveal melanoma is critical for its prognosis and therapy, and it can indicate the metastatic risk of the tumor. Tumor segmentation is also significant for guiding clinical diagnosis, as it provides morphological and other essential information for clinicians. However, precise uveal melanoma segmentation and basal diameter prediction remain challenging for current computer-aided methods. The scarcity of large, annotated datasets and appropriate multi-task models hinders further exploration to simultaneously and accurately segment the tumors and predict their basal diameter for uveal melanoma. To address this challenge, we collected a novel dataset and proposed the Tumor Segmentation and Basal Diameter Prediction Network for Uveal Melanoma (TSBPNet-UM), which utilizes readily accessible fundus images to accomplish both tasks concurrently. Its two sub-networks, the Uveal Melanoma Segmentation and Basal Diameter Prediction networks, are designed for mutual performance enhancement. The Uveal Melanoma Segmentation network transfers spatial segmentation information to assist the Basal Diameter Prediction network. Conversely, the Basal Diameter Prediction network updates all parameters using basal diameter-derived gradient information. This model effectively overcomes the limitation that conventional models often struggle to converge on the direct prediction task of basal diameter. The superior performance and efficacy of the TSBPNetUM have been demonstrated through extensive experiments.

Retinal images play a crucial role in diagnosing various eye diseases. However, due to the shortage of ophthalmologists, many patients do not receive timely diagnosis and treatment. Though automatic eye disease diagnosis has been improved due to recent advancements in deep learning, current deep learning models often exhibit unsatisfactory performance when applied to multi-labeled ocular disease classification tasks with fundus images. This paper addresses the limitations of existing models by proposing a new retinopathy dataset, the Resk dataset, and the Multi-Resolution Attention Feature Fusion (MRAFF) architecture. The MRAFF modifies current backbones with multiple downsample steps and incorporates three types of attention mechanisms: channel attention, spatial attention, and linear attention. Extensive experiments demonstrate that MRAFF-modified models exhibit improved feature extraction and classification abilities in handling ocular disease screening. The MRAFF is expected to enhance ocular disease diagnosis and may also suit other multi-label disease screenings, thereby inspiring future computer-aided diagnostic methods.

Uveal Melanoma(UM) is a highly aggressive ocular malignancy. Once metastasis occurs, the survival period is very short. An effective prognosis for UM is necessary. The BAP1 gene and Somatic Copy Number Alterations (SCNA) are two of the important indicators for clinical UM prognosis. However, current methods for detecting prognosis-related genes have limitations, including high costs and the need for advanced technical expertise, which makes them challenging to apply in routine clinical practice. Low-cost and efficient prognosis-related gene detection is worth studying. In this paper, we investigate the prognosis-related gene subtyping prediction problem based on Whole Slide Images(WSI). We propose a novel method, UMGen, for WSI-based UM prognosis-related gene subtyping prediction. Specifically, UMGen consists of a multi-region sampling module, a classifier module, and a joint decision module. The classifier module, SAGNet, applies spatial attention to extract multi-scale features. To improve accuracy and generalization, the multi-region joint decision is introduced. Comprehensive quantitative ablation experiments demonstrate that both our UMGen and SAGNet can surpass other competitors and have excellent generalization and robustness. The code and models will be released to the public for further research.

Tabular data is one of the most common forms of data in real-life applications. In tabular data modelling, methods based on Gradient Boosting Decision Trees (GBDT) and neural networks have demonstrated unique advantages on different datasets. However, existing neural network methods still face bottlenecks in processing continuous numerical features. This study revisits the structural design of numerical embedding in tabular modelling based on neural networks to overcome these bottlenecks. We conceptually decouple the numerical embedding process into three core functional modules: numerical augmentation, normalization, and encoding. Based on this design, we have developed three novel models–TabMLPNet, TabKANet, and TabKANet-PLNL. These models apply batch normalization to numerical features and feed them into a single, shared Multi-Layer Perceptrons (MLP) or Kolmogorov-Arnold networks (KAN) encoder. We propose a Piecewise Learnable Noise Layer (PLNL), which enriches data representation by introducing piecewise noise into numerical features, further enhancing the model’s generalization ability. Extensive experiments on 18 public datasets demonstrate that our models achieve significant performance improvements, outperforming other neural network models. This study underscores the significant benefits of integrating Kolmogorov-Arnold Network (KAN) with batch normalization. This combination not only enables efficient normalization of numerical features, but also allows dynamic learning of numerical distributions within each batch. As a result, it effectively mitigates performance bottlenecks that are often caused by numerical skewness. Additionally, we have confirmed that introducing augmented noise during the training process can enhance the model’s feature learning capabilities. Our implementation is publicly available on GitHub and can be accessed at https://github.com/AI-thpremed/TabKANet .

Early asymmetry (AS) and architectural distortion (AD) detection on mammograms are essential in breast cancer diagnosis. However, they are challenging as the prevalence of AS and AD is very low. This paper proposes an efficient AsyDisNet for the AS and AD detection. First, a novel angle-based quadruplet loss is proposed to detect the AS and AD with limited pixel-level labeled mammograms. Second, we scale the AsyDisNet with a novel semi-weakly supervised learning framework to boost the detection performance with a large number of mammograms with image-level labels extracted from medical reports. The validation on the two largest and privately collected datasets shows an average of % improvement over State-of-the-Art baselines in terms of sensitivities under various false-positive-per-image (FPPI). Furthermore, the proposed AsyDisNet is scalable to the current Picture Archiving and Communication System (PACS) with incremental learning ability. The dataset will be made publicly available at https://github.com/ML-AILab/AsyDisNet.

Vision computer-aided diagnostic methods have been used in early ophthalmic disease screening and diagnosis. However, the limited output formats of these methods lead to poor human-computer interaction and low clinical applicability value. Thus, ophthalmic visual question answering is worth studying. Unfortunately, no practical solutions exist before Large Language Models(LLMs). In this paper, we investigate the ophthalmic visual diagnostic interaction problem. We construct an ophthalmology large language-and-vision assistant, OphGLM, consisting of an image encoder, a text encoder, a fusion module, and an LLM module. We establish a new Chinese ophthalmic fine-tuning dataset, FundusTuning-CN, including the fundus instruction and conversation sets. Based on FundusTuning-CN, we establish a novel LLM-tuning strategy to introduce visual model understanding and ophthalmic knowledge into LLMs at a low cost and high efficiency. Leveraging the pre-training of the image encoder, OphGLM demonstrates strong visual understanding and surpasses open-source visual language models in common fundus disease classification tasks. The FundusTuning-CN enables OphGLM to surpass open-source medical LLMs in both ophthalmic knowledge and interactive capabilities. Our proposed OphGLM has the potential to revolutionize clinical applications in ophthalmology. The dataset, code, and models will be publicly available at https://github.com/ML-AILab/OphGLM.

In this paper, we investigate the benefits of self-supervised learning (SSL) to downstream tasks of satellite images. Unlike common student academic projects, this work focuses on the advantages of the SSL for deployment-driven tasks which have specific scenarios with low or high-spatial resolution images. Our preliminary experiments demonstrate the robust benefits of the SSL trained by medium-resolution (10m) images to both low-resolution (100m) scene classification case (4.25%↑) and very high-resolution (5cm) aerial image segmentation case (1.96%↑), respectively.

Ophthalmologists have used fundus images to screen and diagnose eye diseases. However, different equipments and ophthalmologists pose large variations to the quality of fundus images. Low-quality (LQ) degraded fundus images easily lead to uncertainty in clinical screening and generally increase the risk of misdiagnosis. Thus, real fundus image restoration is worth studying. Unfortunately, real clinical benchmark has not been explored for this task so far. In this paper, we investigate the real clinical fundus image restoration problem. Firstly, We establish a clinical dataset, Real Fundus (RF), including 120 low- and high-quality (HQ) image pairs. Then we propose a novel Transformer-based Generative Adversarial Network (RFormer) to restore the real degradation of clinical fundus images. The key component in our network is the Window-based Self-Attention Block (WSAB) which captures non-local self-similarity and long-range dependencies. To produce more visually pleasant results, a Transformer-based discriminator is introduced. Extensive experiments on our clinical benchmark show that the proposed RFormer significantly outperforms the state-of-the-art (SOTA) methods. In addition, experiments of downstream tasks such as vessel segmentation and optic disc/cup detection demonstrate that our proposed RFormer benefits clinical fundus image analysis and applications.