Yiqun Wang, Chaoqun Wan, Sile Hu, Yonggang Zhang, Xiang Tian, Yaowu Chen, Xu Shen, Jieping Ye. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Task arithmetic is a straightforward yet highly effective strategy for model merging, enabling the resultant model to exhibit multi-task capabilities. Recent research indicates that models demonstrating linearity enhance the performance of task arithmetic. In contrast to existing methods that rely on the global linearization of the model, we argue that this linearity already exists within the model's submodules. In particular, we present a statistical analysis and show that submodules (e.g., layers, self-attentions, and MLPs) exhibit significantly higher linearity than the overall model. Based on these findings, we propose an innovative model merging strategy that independently merges these submodules. Especially, we derive a closed-form solution for optimal merging weights grounded in the linear properties of these submodules. Experimental results demonstrate that our method consistently outperforms the standard task arithmetic approach and other established baselines across different model scales and various tasks. This result highlights the benefits of leveraging the linearity of submodules and provides a new perspective for exploring solutions for effective and practical multi-task model merging.

Brain-machine interfaces (BMIs) have been an important research area in closed-loop neuroscience and neuroengineering. In real-time neuroscience applications, many issues require special consideration, such as trial variability, spike sorting noise or multi-unit activity. For a BMI application of detecting acute pain signals, we discuss several practical issues in BMI applications and propose a new approach for change-point detection based on ensembles of independent detectors. Motivated from unsupervised ensemble learning, the “ensembles of change-point detectors” (ECPDs) combine the decision results from multiple independent detectors, which may be trained from data recorded at different trials or derived from different methodologies. The goal of ECPDs is to reduce the detection error (in terms of false negative and false positive rates) in online BMI applications. We validate our method using computer simulations and experimental recordings from freely behaving rats.

Human facial morphology is a combination of many complex traits. Little is known about the genetic basis of common facial morphological variation. Existing association studies have largely used simple landmark-distances as surrogates for the complex morphological phenotypes of the face. However, this can result in decreased statistical power and unclear inference of shape changes. In this study, we applied a new image registration approach that automatically identified the salient landmarks and aligned the sample faces using high density pixel points. Based on this high density registration, three different phenotype data schemes were used to test the association between the common facial morphological variation and 10 candidate SNPs, and their performances were compared. The first scheme used traditional landmark-distances; the second relied on the geometric analysis of 15 landmarks and the third used geometric analysis of a dense registration of ∼30,000 3D points. We found that the two geometric approaches were highly consistent in their detection of morphological changes. The geometric method using dense registration further demonstrated superiority in the fine inference of shape changes and 3D face modeling. Several candidate SNPs showed potential associations with different facial features. In particular, one SNP, a known risk factor of non-syndromic cleft lips/palates, rs642961 in the IRF6 gene, was validated to strongly predict normal lip shape variation in female Han Chinese. This study further demonstrated that dense face registration may substantially improve the detection and characterization of genetic association in common facial variation.