Alternative polyadenylation (APA) is an important driver of transcriptome diversity that generates messenger RNA isoforms with distinct 3' ends. The rapid development of single-cell and spatial transcriptomic technologies opened up new opportunities for exploring APA data to discover hidden cell subpopulations invisible in conventional gene expression analysis. However, conventional gene-level analysis tools are not fully applicable to APA data, and commonly used unsupervised dimensionality reduction methods often disregard experimentally derived annotations such as cell type identities. Here, we proposed a supervised analytical framework termed spvAPA, specifically used for APA analysis from both single-cell and spatial transcriptomics data. First, an iterative imputation method based on weighted nearest neighbor was designed to recover missing APA signatures, by integrating both gene expression and APA modalities. Second, a supervised feature selection method based on sparse partial least squares discriminant analysis was devised to identify APA features distinguishing cell types or spatial morphologies. Additionally, spvAPA improves the visualization of high-dimensional data for discovering novel cell subtypes, which considers APA features and dual modalities of gene expression and APA. Evaluations across nine single-cell and spatial transcriptomics datasets demonstrate the effectiveness and applicability of spvAPA. spvAPA is available at https://github.com/BMILAB/spvAPA.

Autism Spectrum Disorder (ASD) is a complex and heterogeneous disorder, which has been reported affected by many different causative genes. In recent years, single-nucleus RNA sequencing (snRNA-seq) have been widely used to reveal variation in gene expression across cell types during the pathology of ASD. However, the vast majority of current studies only consider information on the single modality of gene expression profile obtained from snRNA-seq. Recent bioinformatics studies have successfully captured key transcriptomic information on alternative polyadenylation (APA) from single-cell RNA-seq (scRNA-seq) data to reveal APA dynamics between cell types. Here we integrated multimodal information of gene expression and APA to gain a more comprehensive understanding of the molecular mechanisms of ASD. APA sites were first identified at the single cell level in patients with ASD and healthy populations under different cell types. Gene features were selected by two unsupervised methods using profiles of APA usages and gene expression, respectively. Then these gene features were integrated to build a cell-type-specific classification model based on multiview privileged support vector machines to predict whether a cell is from a ASD patient or a healthy control. With this model, the contribution of an individual modality was calculated, and genes from different modalities were ranked by importance score to identify high-risk genes that are highly associated with ASD. The functions of these high-risk genes were found to be associated with ASD-related biological functions and significantly different across cell types, highlighting the cell-type-specificity of ASD. These high confidence risk genes can serve as potential biomarkers for the diagnosis and treatment of ASD.

Food recognition plays an important role in food choice and intake, which is essential to the health and well-being of humans. It is thus of importance to the computer vision community, and can further support many food-oriented vision and multimodal tasks, e.g., food detection and segmentation, cross-modal recipe retrieval and generation. Unfortunately, we have witnessed remarkable advancements in generic visual recognition for released large-scale datasets, yet largely lags in the food domain. In this paper, we introduce Food2K, which is the largest food recognition dataset with 2,000 categories and over 1 million images. Compared with existing food recognition datasets, Food2K bypasses them in both categories and images by one order of magnitude, and thus establishes a new challenging benchmark to develop advanced models for food visual representation learning. Furthermore, we propose a deep progressive region enhancement network for food recognition, which mainly consists of two components, namely progressive local feature learning and region feature enhancement. The former adopts improved progressive training to learn diverse and complementary local features, while the latter utilizes self-attention to incorporate richer context with multiple scales into local features for further local feature enhancement. Extensive experiments on Food2K demonstrate the effectiveness of our proposed method. More importantly, we have verified better generalization ability of Food2K in various tasks, including food image recognition, food image retrieval, cross-modal recipe retrieval, food detection and segmentation. Food2K can be further explored to benefit more food-relevant tasks including emerging and more complex ones (e.g., nutritional understanding of food), and the trained models on Food2K can be expected as backbones to improve the performance of more food-relevant tasks. We also hope Food2K can serve as a large scale fine-grained visual recognition benchmark, and contributes to the development of large scale fine-grained visual analysis.

Optimising the approximation of Average Precision (AP) has been widely studied for image retrieval. Limited by the definition of AP, such methods consider both negative and positive instances ranking before each positive instance. However, we claim that only penalizing negative instances before positive ones is enough, because the loss only comes from these negative instances. To this end, we propose a novel loss, namely Penalizing Negative instances before Positive ones (PNP), which can directly minimize the number of negative instances before each positive one. In addition, AP-based methods adopt a fixed and sub-optimal gradient assignment strategy. Therefore, we systematically investigate different gradient assignment solutions via constructing derivative functions of the loss, resulting in PNP-I with increasing derivative functions and PNP-D with decreasing ones. PNP-I focuses more on the hard positive instances by assigning larger gradients to them and tries to make all relevant instances closer. In contrast, PNP-D pays less attention to such instances and slowly corrects them. For most real-world data, one class usually contains several local clusters. PNP-I blindly gathers these clusters while PNP-D keeps them as they were. Therefore, PNP-D is more superior. Experiments on three standard retrieval datasets show consistent results with the above analysis. Extensive evaluations demonstrate that PNP-D achieves the state-of-the-art performance. Code is available at https://github.com/interestingzhuo/PNPloss

Recognizing the category and its ingredient composition from food images facilitates automatic nutrition estimation, which is crucial to various health relevant applications, such as nutrition intake management and healthy diet recommendation. Since food is composed of ingredients, discovering ingredient-relevant visual regions can help identify its corresponding category and ingredients. Furthermore, various ingredient relationships like co-occurrence and exclusion are also critical for this task. For that, we propose an ingredient-oriented multi-task food category-ingredient joint learning framework for simultaneous food recognition and ingredient prediction. This framework mainly involves learning an ingredient dictionary for ingredient-relevant visual region discovery and building an ingredient-based semantic-visual graph for ingredient relationship modeling. To obtain ingredient-relevant visual regions, we build an ingredient dictionary to capture multiple ingredient regions and obtain the corresponding assignment map, and then pool the region features belonging to the same ingredient to identify the ingredients more accurately and meanwhile improve the classification performance. For ingredient-relationship modeling, we utilize the visual ingredient representations as nodes and the semantic similarity between ingredient embeddings as edges to construct an ingredient graph, and then learn their relationships via the graph convolutional network to make label embeddings and visual features interact with each other to improve the performance. Finally, fused features from both ingredient-oriented region features and ingredient-relationship features are used in the following multi-task category-ingredient joint learning. Extensive evaluation on three popular benchmark datasets (ETH Food-101, Vireo Food-172 and ISIA Food-200) demonstrates the effectiveness of our method. Further visualization of ingredient assignment maps and attention maps also shows the superiority of our method.