Recently, model merging methods have demonstrated powerful strengths in combining abilities on various tasks from multiple Large Language Models (LLMs). While previous model merging methods mainly focus on merging homogeneous models with identical architecture, they meet challenges when dealing with Multimodal Large Language Models (MLLMs) with inherent heterogeneous property, including differences in model architecture and the asymmetry in the parameter space. In this work, we propose AdaMMS1, a novel model merging method tailored for heterogeneous MLLMs. Our method tackles the challenges in three steps: mapping, merging and searching. Specifically, we first design mapping function between models to apply model merging on MLLMs with different architecture. Then we apply linear interpolation on model weights to actively adapt the asymmetry in the heterogeneous MLLMs. Finally in the hyper-parameter searching step, we propose an unsupervised hyper-parameter selection method for model merging. As the first model merging method capable of merging heterogeneous MLLMs without labeled data, extensive experiments on various model combinations demonstrated that AdaMMS outperforms previous model merging methods on various vision-language benchmarks.2

Incorporating external knowledge into dialogue generation (DG) is crucial for enhancing response accuracy, where evidence fragments serve as effective knowledgeable snippets that support factual dialogue replies. However, introducing irrelevant content beyond valid knowledge fragments can adversely affect reply quality and lead to hallucinated responses. Prior work relies on manual annotations to develop models for identifying evidence within external knowledge. However, these annotations often cover only a limited portion of the valid evidence, restricting the ability of models to mine useful evidence from retrieved knowledge. To fully Unleash the potential of evidence, we propose a framework to effectively incorporate Evidence in knowledge-Intensive Dialogue Generation (U-EIDG). Specifically, we develop an evidence miner (Evid-M) that harnesses the power of large language models (LLMs) to mine reliable evidence labels from external knowledge. Subsequently, we propose an evidence indicator (Evid-I) to effectively identify valid evidence from retrieved knowledge by utilizing these evidence labels. Furthermore, we introduce an evidence-augmented generator (EAG) incorporating an evidence-attention mechanism that enables the model to focus on segments supported by evidence. Experimental results on the MultiDoc2Dial and WoW benchmarks indicate that the proposed method significantly outperforms other baselines, with a +3∼5 points improvement in Rouge-L. Further analysis confirms the effectiveness of fully mining valid evidence fragments for knowledge-intensive dialogue generation.

Technical debt refers to suboptimal solutions adopted for short-term goals. Self-admitted technical debt (SATD) is the debt that is explicitly marked through comments or documentation, making it traceable. Multi-classification of SATD helps developers understand different debt types and improve efficiency. This paper proposes a SATD multi-classification method based on Fine-Tuning the GPT-3.5-turbo model for SATD prediction. This study uses a public dataset containing 10 projects with code comments. We classify design debt, requirement debt, and defect debt and evaluate our method’s performance. The experimental results show that compared to the best baseline model, our method achieves average improvements of 11.41%, 1.72% and 3.72% in MacroF, MacroP and MacroR metrics, respectively, in the MTO scenario. In the OTO scenario, improvements are 2.33%, 3.70% and 2.18%, respectively. These results indicate that our method has a strong generalization ability in SATD multi-classification and offers a new approach to managing technical debt.

Chi Chen, Yiyang Du, Zheng Fang, Ziyue Wang, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

In this paper, we introduce an ultra-low power strain sensing system with high sensitivity and scalability for next generation intelligent wearable applications. Firstly, we demonstrate a simple infiltration method to transfer laser induced graphene (LIG) strain sensor onto a PDMS flexible substrate to prepare µm level thin film, and its three-dimensional strain characteristics were studied for the first time which is closer to the actual human movements sensing situation. Secondly, to enhance its integration into wearable devices, we have developed an ultra-low-power readout circuit tailored to the sensor’s resistance range. A novel direct-digital time-domain readout technique with a pulse width locked loop and a time-to-digital converter is proposed. Leveraging the LIG/PDMS platform, our strain sensor showcases exceptional sensitivity (GF≈225.1) within a 0-22% strain range and impressive linearity (R2=0.99062). Implemented in 65nm CMOS technology, the readout circuit achieves a 10-bit resolution as well as a rapid conversion time of 0.3ms. Remarkably, the overall sensing system features a low power consumption of merely 5µW including both the sensor and circuits. The exceptional features of the LIG sensor and its associated readout circuit make them highly promising candidates for integration into wearable smart electronic devices designed for healthcare applications and human movement tracking.

In the spring of 2020, lockdown measures were adopted to control the spread of novel coronavirus, and these measurements had large impacts on air quality, regional climate, and vegetation growth. Accurately capturing the response of vegetation to environmental changes is crucial for ecosystem research. However, there might be some divergences in vegetation growth trends from different satellite datasets. To investigate the response of vegetation growth to environmental changes from January to April 2020, we employed two reconstructed solar-induced chlorophyll fluorescence (SIF) products, continuous SIF (CSIF) and global OCO-2 SIF (GOSIF), normalized difference vegetation index (NDVI), enhanced vegetation index (EVI), and leaf area index (LAI). Results shows that CSIF, GOSIF, NDVI, EVI, and LAI captured enhanced vegetation growth during January–March 2020 compared with 2015–2019. However, CSIF and GOSIF show an opposite trend to vegetation indices (VIs) and LAI in April. The trends from CSIF and GOSIF also differed with those from TROPOspheric Monitoring Instrument (TROPOMI) SIF and OCO-2 SIF. Biases were attributed to the input MODIS surface reflectance data for reconstructing SIF in the algorithms of GOSIF and CSIF.