Foundation models have made significant strides in understanding the genomic language of DNA sequences. However, previous models typically adopt the tokenization methods designed for natural language, which are unsuitable for DNA sequences due to their unique characteristics. In addition, the optimal approach to tokenize DNA remains largely under-explored, and may not be intuitively understood by humans even if discovered. To address these challenges, we introduce MxDNA, a novel framework where the model autonomously learns an effective DNA tokenization strategy through gradient decent. MxDNA employs a sparse Mixture of Convolution Experts coupled with a deformable convolution to model the tokenization process, with the discontinuous, overlapping, and ambiguous nature of meaningful genomic segments explicitly considered. On Nucleotide Transformer Benchmarks and Genomic Benchmarks, MxDNA demonstrates superior performance to existing methods with less pretraining data and time, highlighting its effectiveness. Finally, we show that MxDNA learns unique tokenization strategy distinct to those of previous methods and captures genomic functionalities at a token level during self-supervised pretraining. Our MxDNA aims to provide a new perspective on DNA tokenization, potentially offering broad applications in various domains and yielding profound insights. Code is available at https://github.com/qiaoqiaoLF/MxDNA.

RNA plays a pivotal role in translating genetic instructions into functional outcomes, underscoring its importance in biological processes and disease mechanisms. Despite the emergence of numerous deep learning approaches for RNA, particularly universal RNA language models, there remains a significant lack of standardized benchmarks to assess the effectiveness of these methods. In this study, we introduce the first comprehensive RNA benchmark BEACON BEnchmArk for COmprehensive RNA Task and Language Models).First, BEACON comprises 13 distinct tasks derived from extensive previous work covering structural analysis, functional studies, and engineering applications, enabling a comprehensive assessment of the performance of methods on various RNA understanding tasks. Second, we examine a range of models, including traditional approaches like CNNs, as well as advanced RNA foundation models based on language models, offering valuable insights into the task-specific performances of these models. Third, we investigate the vital RNA language model components from the tokenizer and positional encoding aspects. Notably, our findings emphasize the superiority of single nucleotide tokenization and the effectiveness of Attention with Linear Biases (ALiBi) over traditional positional encoding methods. Based on these insights, a simple yet strong baseline called BEACON-B is proposed, which can achieve outstanding performance with limited data and computational resources. The datasets and source code of our benchmark are available at https://github.com/terry-r123/RNABenchmark.

3-D multiobject tracking (MOT) is an important task in numerous applications, including robotics and autonomous driving. Nevertheless, existing 3-D MOT solutions suffer from significant performance degradation under adverse weather conditions. Inspired by the fact that hard objects (e.g., missed detections or wrongly-associated objects) are more constructive to performance improvement, in this article, we leverage hard samples for robust 3-D MOT in adverse weather conditions. Specifically, we implement a cross-modality 3-D MOT framework to learn the 3-D region proposals from point clouds and RGB images, respectively. To minimize the risk of missed detection and wrong association, we introduce an adaptive hard sample mining scheme to align the 3-D region proposals provided by two modalities. We quantify the hard level by comparing the confidence values of the same object in the two branches and their distance in the embedding space. Meanwhile, we dynamically adjust the weights of hard samples during training to enhance the representation learning for robust 3-D MOT. Extensive experimental results showcase that our proposed solution effectively mitigates the missed detection and reduces wrong association with good generalization.

Automatic segmentation of fluorescein leakage in fundus fluorescein angiography images is important in the clinical diagnosis of advanced diabetic retinopathy. Despite the recent success of deep-learning-based models in improving medical image segmentation, segmentation of fluorescein leakage has been ignored owing to (1) a lack of publicly available data with sufficient annotations for training a segmentation network and (2) incapability of supervised models to accurately localize fluorescein leakage at different imaging angles. To address these issues, we studied the automatic segmentation of fluorescein leakage in fundus fluorescein angiography images and devised a method involving (1) a cross-modal learning framework for fluorescein leakage segmentation using both image and text data, (2) a dual attention learning module for identifying important linguistic and visual features, and (3) fluorescein-related-keyword classification for identifying meaningful textual expressions pertaining to the location and type of fluorescein leakage. We demonstrate the effectiveness of the proposed method for an in-house fundus fluorescein angiography image data set.

Accurate optic disc segmentation plays an essential role in the early diagnosis of glaucoma, which has been a major cause of irreversible blindness for the past decade. Recently, U-shape Convolutional Neural Network (CNN) models have achieved favourable performance in optic disc segmentation. However, it is worth noting that these models require a large number of pixel-level annotations while these annotations are difficult to obtain in clinical practice. As a solution, weakly-supervised training methods are commonly implemented, but it will provoke U-shape CNN generating inaccurate, diluted, and grid-like segmentation results. In this paper, we propose a novel Hybrid Network (HyNet) to solve the issue above. HyNet consists of a U-shape backbone hybridized with a cross-scale connection structure, which makes better use of multi-scale visual semantics. Nevertheless, the generalization ability of HyNet is affected by the domain shift among different datasets. Therefore, we innovatively combine weakly- and fully-supervised training methods, namely Hybrid Process (HyProcess), to solve the domain shift problem. Experimental results on ONHSD, DRIONS-DB, and DRISHTI-GS datasets show that our model outperforms the state-of-the-art, reaching Dice of 82.39(%), 93.72(%), and 95.34(%) respectively. Additionally, our ablation study validates the effectiveness of HyNet along with HyProcess, and further analysis reflects their value in clinical practice.

Color fundus imaging is an important modality used for ophthalmic disease screening and provides a non-invasive invivo method for assessing body condition. We aimed to assess deep learning models for age prediction from fundus images of normal patients and patients with ophthalmic diseases. In addition, we sought to investigate interpretable clues regarding the salient regions between normal and pathological changes as determined by deep learning models during age prediction. In this study, we used a convolutional neural network model for age prediction and evaluated it on an in-house database of fundus images of the Chinese population. The results of the experiment revealed some conclusions as follows: (1) deep learning-based classification models have better age prediction performance than deep learning-based regression models of fundus images; (2) deep learning-based models tend to use holistic information of the fundus for age prediction; (3) ophthalmic diseases that cause damage to the structure of the fundus and change its appearance will result in a decline in age prediction performance.

While fundus image interpretation and report writing are routine procedures in the diagnosis of ophthalmic diseases, they are error-prone for inexperienced ophthalmologists and laborious and tedious for experienced ophthalmologists. Therefore, there is an urgent need for an automated ophthalmic report generation tool. Existing methods for captioning medical images are confined to the field of radiological images, and there is no such method for fundus images. To address this issue, we studied automated report generation for fundus images. Unlike the radiological field, this task presents several challenges. First, there are a wide variety of ophthalmic diseases, and identifying diseases in fundus images is sometimes difficult since the diseases can have different manifestations. Second, an ophthalmic disease may have multiple subtypes, which can be distinguished from one another only through accurate identification o f specific ophthalmic symptoms. To overcome these challenges, we propose a method with the following features: (1) a multitask learning framework for predicting both ophthalmic symptoms and pathologies and for generating paragraphs, (2) two-branch architecture for handling ophthalmic symptoms and pathologies separately, and (3) a causal linkage module for transferring the symptom information for accurately predicting ophthalmic pathologies and thereby facilitating the identification of subtypes. We demonstrate the effectiveness of the proposed method for an in-house fundus image data set.

Automatic detection of chorioretinopathy plays an important role in clinical practice, but the detection of a major chorioretinopathy of central serous chorioretinopathy based on fundus photography images has rarely been studied, let alone distinguishing it from another chorioretinopathy of central exudative chorioretinopathy. Due to the high degree of similarity between the two chorioretinopathies on fundus images, it is difficult for the latest automatic methods to accurately distinguish between them. In this study, we design a deep neural network with two branches for different classification tasks, where the first one is to distinguish the normal and abnormal while the other is to classify the two chorioretinopathies. We manage to improve the classification accuracy by combining focal loss and discriminative loss. Extensive experiments are conducted for comparison between our method and other universal classification models using a private retinal fundus dataset. The results demonstrate that our method achieves the best performance with 97.69%, 99.58% and 98.87% on the accuracy, precision and sensitivity, respectively.