Recent advances in large language models (LLMs) have shown remarkable capabilities across textual and multimodal domains. In parallel, large language diffusion models have emerged as a promising alternative to the autoregressive paradigm, offering improved controllability, bidirectional context modeling, and robust generation. However, their application to the audio modality remains underexplored. In this work, we introduce DIFFA, the first diffusion-based large audio-language model designed to perform spoken language understanding. DIFFA integrates a frozen diffusion language model with a lightweight dual-adapter architecture that bridges speech understanding and natural language reasoning. We employ a two-stage training pipeline: first, aligning semantic representations via an ASR objective; then, learning instruction-following abilities through synthetic audio-caption pairs automatically generated by prompting LLMs. Despite being trained on only 960 hours of ASR and 127 hours of synthetic instruction data, DIFFA demonstrates competitive performance on major benchmarks, including MMSU, MMAU, and VoiceBench, outperforming several autoregressive open-source baselines. Our results reveal the potential of large language diffusion models for efficient and scalable audio understanding, opening a new direction for speech-driven AI.

Recent advances in speech large language models (e.g., GPT-4o) have enabled end-to-end spoken interactions, yet their robustness in realworld applications remains unclear, where systems must assist users in completing specific tasks under complex conditions such as multiturn, ambiguous, and often spontaneous speech, as well as natural alternation between speech and text.Task-oriented dialogue (TOD) offers a realistic scenario to evaluate whether models can effectively help users accomplish such task-oriented goals, but existing benchmarks are mainly text-based, and the few speech datasets are limited to English and often neglect spontaneous disfluencies and speaker diversity.To address this gap, we introduce RealTalk-CN, the first Chinese multi-turn, multi-domain speech-text TOD dataset, containing 5.4k dialogues (60K turns, ~150 hours) of real human-to-human recordings with detailed annotations for dialogue states, disfluency types, and speaker characteristics.Based on this dataset, we propose a cross-modal interaction task supporting dynamic speech-text switching and a comprehensive evaluation protocol assessing robustness to disfluencies, sensitivity to speaker variation, and cross-domain generalization.Experiments on state-of-the-art models demonstrate the challenges posed by RealTalk-CN and establish its value as a benchmark for developing reliable and fair Speech LLMs in real-world deployments.The dataset and evaluation framework are available 1 to encourage further research.

This paper addresses a problem of event-triggered state estimation for a linear time-varying Gaussian system over the Internet of Underwater Things (IoUT), where the IoUT is a hybrid topology including underwater acoustic wireless sensors and surface wireless network. Distributed state estimation aims to reconstruct the system state using noisy measurements and local neighbor information, both of which are transmitted via the IoUT. However, IoUT is subject to limited energy and communication bandwidth. Each node, especially applied in underwater case, selectively transmits necessary data to maintain a minimum communication load, and thus to improve energy efficiency and prolong network lifetime. To achieve this goal, we design two event-triggered strategies for typical types of wireless communication channels, of which one channel is used to transmit the local posterior information pair of filter and the other is used to transmit measurement of sensor. Then, based on the covariance intersection fusion rule and the event-triggered strategies, we develop a novel consensus-based distributed state estimator with dual event-triggered communication in a recursive form. Further, after guaranteeing the network connectivity and system collective observability constraints, we derive the uniformly mean-square upper bound of the estimation error of each node. Finally, we provide an example about underwater target tracking to illustrate the effectiveness of the proposed approach.

Continual learning aims to learn new tasks sequentially without forgetting previously acquired knowledge. However, catastrophic forgetting remains a significant challenge. In this paper, we introduce kNN-CL, a simple yet effective approach that harnesses k-nearest neighbors (kNN) to mitigate forgetting in continual learning. Specifically, kNN-CL identifies the k most similar instances (key-value pairs) from the previous tasks to refine model predictions, enabling the model to adapt to the relevant task for a given test instance. Notably, kNN-CL can be seamlessly integrated into existing continual learning frameworks in a plug-and-play manner, without any additional training. Modern deep neural networks have achieved remarkable progress on a wide range of tasks. However, they encounter difficulties when processing sequential data streams. As these networks recalibrate their parameters to assimilate new data, they inadvertently compromise on previously acquired knowledge, leading to an issue known as catastrophic forgetting. Experimental results show that kNN-CL substantially improves accuracy in both settings, demonstrating the effectiveness of kNN-CL in mitigating catastrophic forgetting.

Deep learning models for medical disease detection require extensive labeled data, which is often scarce and expensive. Transfer learning can help by leveraging knowledge from large source domains, but directly fine-tuning these models can lead to catastrophic forgetting, making it impossible to reuse it for new diseases that continue to emerge. Current continual learning methods can mitigate catastrophic forgetting but fail to support efficient knowledge transfer and use mutually exclusive classifiers, which are inadequate for multi-disease prediction in medical imaging. To solve these issues, we propose a Continual Learning Multi-Disease Prediction Framework with class-specific adapters to independently model each disease, supporting multi-disease prediction and feature integration. Additionally, we introduce a two-stage knowledge integration method based on self-attention, which concatenates features from previous tasks and computes attention scores to utilize historical knowledge effectively. Experimental results demonstrate that our framework supports multi-disease prediction and knowledge transfer effectively. They also confirm the effectiveness of our self-attention-based method, providing robust support for medical disease detection.

Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, Xiaohang Dong. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.