Retrieval-augmented generation (RAG) is usually integrated into large language models (LLMs) to mitigate hallucinations and knowledge obsolescence.Whereas, conventional onestep retrieve-and-read methods are insufficient for multi-hop question answering, facing challenges of retrieval semantic mismatching and the high cost in handling interdependent subquestions.In this paper, we propose Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answering (Q-DREAM).Q-DREAM consists of three key modules: (1) the Question Decomposition Module (QDM), which decomposes multi-hop questions into fine-grained subquestions; (2) the Subquestion Dependency Optimizer Module (SDOM), which models the interdependent relations of subquestions for better understanding; and (3) the Dynamic Passage Retrieval Module (DPRM), which aligns subquestions with relevant passages by optimizing the semantic embeddings.Experimental results across various benchmarks demonstrate that Q-DREAM significantly outperforms existing RAG methods, achieving state-of-the-art performance in both in-domain and out-of-domain settings.Notably, Q-DREAM also improves retrieval efficiency while maintaining high accuracy compared with recent baselines.

Large language models (LLMs) have shown great success in various Natural Language Processing (NLP) tasks, whist they still need updates after deployment to fix errors or keep pace with the changing knowledge in the world. Researchers formulate such problem as Model Editing and have developed various editors focusing on different axes of editing properties. However, current editors can hardly support all properties and rely on heavy computational resources. In this paper, we propose a plug-in Model Editing method based on neuron-indexed dynamic LoRA (MELO), which alters the behavior of language models by dynamically activating certain LoRA blocks according to the index built in an inner vector database. Our method satisfies various editing properties with high efficiency and can be easily integrated into multiple LLM backbones. Experimental results show that our proposed MELO achieves state-of-the-art editing performance on three sequential editing tasks (document classification, question answering and hallucination correction), while requires the least trainable parameters and computational cost.

Federated Learning (FL) has drawn much attention for distributed system over the Internet of Things (IoT), since it enables collaborative machine learning on heterogeneous devices while resolves concerns about privacy leakage. Due to the catastrophic forgetting (CF) phenomenon of optimization methods, existing FL approaches are restricted to single task learning and typically assume that data from all nodes are simultaneously available during training. However, in practical IoT scenarios, the data preparation from nodes may be asynchronous, and different tasks require incremental training. To address the issues, we propose a continual FL (CFL) framework with parameter-level importance aggregation (PI-Fed), which supports collaborative task-incremental learning with privacy preservation. Specifically, PI-Fed evaluates the importance of each parameter in the global model to all history tasks, which is computed locally and aggregated at the center server. Then the server performs soft-masking on the averaged gradient collected from local clients based on the parameter importance. By minimizing the change on important parameters, PI-Fed effectively overcomes CF and also achieves high efficiency without experience replay. Extensive experiments on 4 benchmarks with at most 20 sequential tasks demonstrate that our proposed PI-Fed significantly outperforms traditional FL baselines (FedAvg, FedNova, and SCAFFOLD).

In the scenario of Model-as-a-Service (MaaS), pre-trained models are usually released as inference APIs. Users are allowed to query those models with manually crafted prompts. Without accessing the network structure and gradient information, it's tricky to perform continuous prompt tuning on MaaS, especially for vision-language models (VLMs) considering cross-modal interaction. In this paper, we propose a black-box prompt tuning framework for VLMs to learn task-relevant prompts without back-propagation. In particular, the vision and language prompts are jointly optimized in the intrinsic parameter subspace with various evolution strategies. Different prompt variants are also explored to enhance the cross-model interaction. Experimental results show that our proposed black-box prompt tuning framework outperforms both hand-crafted prompt engineering and gradient-based prompt learning methods, which serves as evidence of its capability to train task-relevant prompts in a derivative-free manner.

A critical component of competence in language is being able to identify relevant components of an utterance and reply appropriately. In this paper we examine the extent of such dialogue response sensitivity in pre-trained language models, conducting a series of experiments with a particular focus on sensitivity to dynamics involving phenomena of at-issueness and ellipsis. We find that models show clear sensitivity to a distinctive role of embedded clauses, and a general preference for responses that target main clause content of prior utterances. However, the results indicate mixed and generally weak trends with respect to capturing the full range of dynamics involved in targeting at-issue versus not-at-issue content. Additionally, models show fundamental limitations in grasp of the dynamics governing ellipsis, and response selections show clear interference from superficial factors that outweigh the influence of principled discourse constraints.

Deep transformer models have pushed performance on NLP tasks to new limits, suggesting sophisticated treatment of complex linguistic inputs, such as phrases.However, we have limited understanding of how these models handle representation of phrases, and whether this reflects sophisticated composition of phrase meaning like that done by humans.In this paper, we present systematic analysis of phrasal representations in state-of-the-art pre-trained transformers.We use tests leveraging human judgments of phrase similarity and meaning shift, and compare results before and after control of word overlap, to tease apart lexical effects versus composition effects.We find that phrase representation in these models relies heavily on word content, with little evidence of nuanced composition.We also identify variations in phrase representation quality across models, layers, and representation types, and make corresponding recommendations for usage of representations from these models.

In this paper we present VinaSC, an improved version of Autodock Vina, that performs molecular docking simulation efficiently on large-scale heterogeneous cluster for massive docking scenario. Both application and platform optimizations are implemented to fully exploit performance potentials of heterogeneous platforms. Specifically, computation is offloaded to Intel Many Integrated Core (MIC) using Intel Coprocessor Offload Infrastructure (COI) to make host CPU and coprocessor collaborate during docking simulation. Moreover, a dynamic scheduling framework is implemented in VinaSC using MPI and Pthread to leverage heterogeneous resources. Our work makes the following improvements: 1) Compared to original Vina that only supports single-node CPU platform, VinaSC fully utilizes computing resources including CPU and MIC coprocessor. 2) Load unbalance due to the random algorithm and heterogeneous platform is alleviated. 3) Utilization of vector units on MIC is significantly improved. 4) VinaSC scales well on heterogeneous cluster, which enables mass docking using clusters. Experiments on a cluster with 6 CPU+MIC nodes using PDBBIND dataset demonstrate that VinaSC outperforms original Vina by more than 2.3×. In addition, VinaSC maintains scalable performance speedup as the docking scale increases.

Determining optical flow has been a wide field of research for more than 20 years now that has not been solved satisfactorily yet. In this work, we study the influence of a nonlinear smoothing process based on bilateral filtering on a Lucas & Kanade framework for the estimation of optical flow between two image frames. Different confidence measures are used to improve the computation process and detect occlusion and innovation phenomena, explicitly handling discontinuous flow fields. By means of simulations we report that the accuracy can be increased significantly, making this approach interesting for further investigations.