Recently, the pretrain-finetune paradigm has become a cornerstone in various deep learning areas.While in general the pre-trained model would promote both effectiveness and efficiency of downstream tasks fine-tuning, studies have shown that not all knowledge acquired during pre-training is beneficial.Some of the knowledge may actually bring detrimental effects to the fine-tuning tasks, which is also known as negative transfer.To address this problem, graceful forgetting has emerged as a promising approach.The core principle of graceful forgetting is to enhance the learning plasticity of the target task by selectively discarding irrelevant knowledge.However, this approach remains underexplored in the context of generative language models, and it is often challenging to migrate existing forgetting algorithms to these models due to architecture incompatibility.To bridge this gap, in this paper we propose a novel framework, Learning With Forgetting (LWF), to achieve graceful forgetting in generative language models.With Fisher Information Matrix weighting the intended parameter updates, LWF computes forgetting confidence to evaluate selfgenerated knowledge regarding the forgetting task, and consequently, knowledge with high confidence is periodically unlearned during fine-tuning.Our experiments demonstrate that, although thoroughly uncovering the mechanisms of knowledge interaction remains challenging in pre-trained language models, applying graceful forgetting can contribute to enhanced fine-tuning performance.

Facial personalization faces challenges to maintain identity fidelity without disrupting the foundation model's prompt consistency. The mainstream personalization models employ identity embedding to integrate identity information within the attention mechanisms. However, our preliminary findings reveal that identity embeddings compromise the effectiveness of other tokens in the prompt, thereby limiting high prompt consistency and attribute-level controllability. Moreover, by deactivating identity embedding, personalization models still demonstrate the underlying foundation models' ability to control facial attributes precisely. It suggests that such foundation models' knowledge can be leveraged to cure the ill-aligned prompt consistency of personalization models. Building upon these insights, we propose FreeCure, a framework that improves the prompt consistency of personalization models with their latent foundation models' knowledge. First, by setting a dual inference paradigm with/without identity embedding, we identify attributes (e.g., hair, accessories, etc.) for enhancements. Second, we introduce a novel foundation-aware self-attention module, coupled with an inversion-based process to bring well-aligned attribute information to the personalization process. Our approach is training-free, and can effectively enhance a wide array of facial attributes; and it can be seamlessly integrated into existing popular personalization models based on both Stable Diffusion and FLUX. FreeCure has consistently shown significant improvements in prompt consistency across these facial personalization models while maintaining the integrity of their original identity fidelity.

Sketch-based 3D shape retrieval has become an intense topic in the multimedia society. Recent progress in this field can be mainly attributed to well-designed loss functions. However, two essential aspects remain to be considered by previous works: Firstly, the inherent abstract, sparse, and diverse nature of the sketch leads to inevitable label noise, which significantly impairs model efficacy. Secondly, a deficient focus on hard sample mining can lead to model performance degradation. To address these issues, we propose a label self-correction method. Starting from the margin-based loss function, we relate the ground truth class center of the sample to its nearest negative class center. By leveraging the decision boundary, this module bolsters the learning capacity within similar classes, ensuring the separability of the feature space. Extensive experiments on two benchmarks shows that our method surpasses previous state-of-the-art methods.

In recent years, sketch-based 3D shape retrieval has attracted growing attention. While many previous studies have focused on cross-modal matching between hand-drawn sketches and 3D shapes, the critical issue of how to handle low-quality and noisy samples in sketch data has been largely neglected. This paper presents an uncertainty-aware cross-modal transfer network (UACTN) that addresses this issue. UACTN decouples the representation learning of sketches and 3D shapes into two separate tasks: classification-based sketch uncertainty learning and 3D shape feature transfer. We first introduce an end-to-end classification-based approach that simultaneously learns sketch features and uncertainty, allowing uncertainty to prevent overfitting noisy sketches by assigning different levels of importance to clean and noisy sketches. Then, 3D shape features are mapped into the pre-learned sketch embedding space for feature alignment. Extensive experiments and ablation studies on two benchmarks demonstrate the superiority of our proposed method compared to state-of-the-art methods.

Identifying drug phenotypic effects, including therapeutic effects and adverse drug reactions (ADRs), is an inseparable part for evaluating the potentiality of new drug candidates (NDCs). However, current computational methods for predicting phenotypic effects of NDCs are mainly based on the overall structure of an NDC or a related target. These approaches often lead to inconsistencies between the structures and functions and limit the prediction space of NDCs. In this study, first, we constructed quantitative associations of substructure-domain, domain-ADR, and domain-ATC (Anatomical Therapeutic Chemical Classification System code) through L1LOG and L1SVM machine learning models. These associations represent relationships between phenotypes (ADRs and ATCs) and local structures of drugs and proteins. Then, based on these established associations, substructure-phenotype relationships were constructed which were utilized to quantify drug-phenotype relationships. Thus, this approach could achieve high-throughput and effective evaluations of the druggability of NDCs by referring to the established substructure-phenotype relationships and structural information of NDCs without additional prior knowledge. Using this computational pipeline, 83,205 drug-ATC relationships (including 1,479 drugs and 178 ATCs) and 306,421 drug-ADR relationships (including 1,752 drugs and 454 ADRs) were predicted in total. The prediction results were validated at four levels: five-fold cross validation, public databases, literature, and molecular docking. Furthermore, three case studies demonstrated the feasibility of our method. 79 ATCs and 269 ADRs were predicted to be related to Maraviroc, an approved drug, including the existing antiviral effect in clinical use. Additionally, we also found risk substructures of severe ADRs, for example, SUB215 (>= 1, saturated or only aromatic carbon ring size 7) can result in shock. And we analyzed the mechanism of action (MOA) of interested drugs based on the established drug-substructure-domain-protein associations. In a word, this approach through establishing drug-substructure-phenotype relationships can achieve quantitative prediction of phenotypes for a given NDC or drug without any prior knowledge except its structure information. Using that way, we can directly obtain the relationships between substructure and phenotype of a compound, which is more convenient to analyze the phenotypic mechanism of drugs and accelerate the process of rational drug design.

Few-shot font generation (FFG), which aims to generate a new font with a few examples, is gaining increasing attention due to the significant reduction in labor cost. A typical FFG pipeline considers characters in a standard font library as content glyphs and transfers them to a new target font by extracting style information from the reference glyphs. Most existing solutions explicitly disentangle content and style of reference glyphs globally or component-wisely. However, the style of glyphs mainly lies in the local details, i.e. the styles of radicals, components, and strokes together depict the style of a glyph. Therefore, even a single character can contain different styles distributed over spatial locations. In this paper, we propose a new font generation approach by learning 1) the fine-grained local styles from references, and 2) the spatial correspondence between the content and reference glyphs. Therefore, each spatial location in the content glyph can be assigned with the right fine-grained style. To this end, we adopt cross-attention over the representation of the content glyphs as the queries and the representations of the reference glyphs as the keys and values. Instead of explicitly disentangling global or component-wise modeling, the cross-attention mechanism can attend to the right local styles in the reference glyphs and aggregate the reference styles into a fine-grained style representation for the given content glyphs. The experiments show that the proposed method outperforms the state-of-the-art methods in FFG. In particular, the user studies also demonstrate the style consistency of our approach significantly outperforms previous methods.

At present, computational methods for drug repositioning are mainly based on the whole structures of drugs, which limits the discovery of new functions due to the similarities between local structures of drugs. In this article, we, for the first time, integrated the features of chemical-genomics (substructure-domain) and pharmaco-genomics (domain-indication) based on the assumption that drug-target interactions are mediated by the substructures of drugs and the domains of proteins to identify the relationships between substructure-indication and establish a drug-substructure-indication network for predicting all therapeutic effects of tested drugs through only information on the substructures of drugs. In total, 83 205 drug-indication relationships with different correlation scores were obtained. We used three different verification methods to indicate the accuracy of the method and the reliability of the scoring system. We predicted all indications of olaparib using our method, including the known antitumor effect and unknown antiviral effect verified by literature, and we also discovered the inhibitory mechanism of olaparib toward DNA repair through its specific sub494 (o = C-C: C), as it participates in the low synthesis of the poly subfunction of the apoptosis pathway (hsa04210) by inhibiting the Inositol 1,4,5-trisphosphate receptor(s) (ITPRs) and hydrolyzing poly (ADP ribose) polymerases. ElectroCardioGrams of four drugs (quinidine, amiodarone, milrinone and fosinopril) demonstrated the effect of anti-arrhythmia. Unlike previous studies focusing on the overall structures of drugs, our research has great potential in the search for more therapeutic effects of drugs and in predicting all potential effects and mechanisms of a drug from the local structural similarity.