Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, Zhongyu Wei. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Achieving Artificial General Intelligence (AGI) requires AI agents that can not only make strategic decisions but also engage in flexible and meaningful communication. Inspired by Wittgenstein's language game theory, we propose that language agents can learn through in-context interaction rather than traditional multi-stage frameworks that separate decision-making from language expression. Using Werewolf, a social deduction game that tests language understanding, strategic interaction, and adaptability, as a test bed, we develop the Multi-agent Kahneman-Tversky's Optimization (MaKTO). MaKTO engages diverse models in extensive gameplay to generate unpaired desirable and unacceptable responses, then employs KTO to refine the model's decision-making process. In 9-player Werewolf games, MaKTO achieves a 61% average win rate across various models, outperforming GPT-4o and two-stage RL agents by relative improvements of 23.0% and 10.9%, respectively. Notably, MaKTO also demonstrates human-like performance, winning 60% against expert players and showing only 48.9% detectability in Turing-style blind tests. Code and data are available at project page https://reneeye.github.io/MaKTO.html.

Social networks are changing in real-time, and$ k $-triangle counting as a fundamental task in graph analysis is useful for finding meaningful connection patterns in these dynamical graphs. However, the dynamical graphs and the number of$ k $-triangles therein both are sensitive and imply users’ private information. The lasted works applied Local Differential Privacy (LDP) to the snapshot graph, counting simple statistical information. Nevertheless, these LDP-based works ignored the real-time development of social graphs and did not count more complex statistics,$ k $-triangle. To this end, we propose alocally differentiallyprivatek-triangles counting in real-time social graph, namely$\mathtt{LAPKE}$that is the first work to provide edge LDP and guarantee a lower upper bound on the estimation error of the number of$ k $-triangles for dynamical graph models. Thereafter, to further reduce the estimation error, we propose$\mathtt{LAPKE^{+}}$and its extension$\mathtt{LAPKE^{++}}$that do not require a large number of users’ efforts and synchronization. The main intuition of$\mathtt{LAPKE}$,$\mathtt{LAPKE^{+}}$and$\mathtt{LAPKE^{++}}$is sampling$ k $or 2 disjoint users in real-time social graph to construct the key substructure$ k $-wedge of$ k $-triangle, which minimizes the estimation error while preserving users’ private information. Moreover, we theoretically prove the lower upper bound on the estimation error of$ k $-triangle for dynamical graphs$\mathtt{LAPKE}$,$\mathtt{LAPKE^{+}}$and$\mathtt{LAPKE^{++}}$provide. Finally, the extensive experiment results on three real-world datasets and one synthetic dataset validate the superior performance of the proposed three algorithms compared with three latest existing methods.

In crowded scenarios, face mask detection algorithms still suffer from the target misdetection and omission, and the difficulty of reconciling real time and effectiveness. To address these issues, we present an G-YOLOv5 face mask detection algorithm. First, we adopt a weighted bidirectional feature pyramid network as the feature fusion network to enhance multiscale feature fusion; second, we utilize the WIoU loss function to strengthen the impact of good anchor frames while weakening the detrimental impact of poor quality anchor frames. Finally, considering the real-time and effectiveness issues of detector, we designed the Ghostv2-C3 module instead of the C3 module of the backbone to improve the inference speed of the model. The results of the experiment demonstrates that compared with other detectors, our detector plays a positive role in solving the target misdetection and omission and balancing the real-time nature of the model.

End-to-end Speech Translation (E2E ST) aims to directly translate source speech into target text.Existing ST methods perform poorly when only extremely small speech-text data are available for training.We observe that an ST model's performance closely correlates with its embedding similarity between speech and source transcript.In this paper, we propose Word-Aligned COntrastive learning (WACO), a simple and effective method for extremely low-resource speech-to-text translation.Our key idea is bridging word-level representations for both speech and text modalities via contrastive learning.We evaluate WACO and other methods on the MuST-C dataset, a widely used ST benchmark, and on a low-resource direction Maltese-English from IWSLT 2023.Our experiments demonstrate that WACO outperforms the best baseline by 9+ BLEU points with only 1-hour parallel ST data.

Jingcong Liang, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Counter-argument generation-a captivating area in computational linguistics-seeks to craft statements that offer opposing views.While most research has ventured into paragraph-level generation, sentence-level counter-argument generation beckons with its unique constraints and brevity-focused challenges.Furthermore, the diverse nature of counter-arguments poses challenges for evaluating model performance solely based on ngram-based metrics.In this paper, we present the ArgTersely benchmark for sentence-level counter-argument generation, drawing from a manually annotated dataset from the Change-MyView debate forum 1 .We also propose Arg-LlaMA for generating high-quality counterargument.For better evaluation, we trained a BERT-based evaluator Arg-Judge with human preference data.We conducted comparative experiments involving various baselines such as LlaMA, Alpaca, GPT-3, and others.The results show the competitiveness of our proposed framework and evaluator in counter-argument generation tasks.

Recently, speech-to-text translation has attracted more and more attention and many studies have emerged rapidly. In this paper, we present a comprehensive survey on direct speech translation aiming to summarize the current state-of-the-art techniques. First, we categorize the existing research work into three directions based on the main challenges --- modeling burden, data scarcity, and application issues. To tackle the problem of modeling burden, two main structures have been proposed, encoder-decoder framework (Transformer and the variants) and multitask frameworks. For the challenge of data scarcity, recent work resorts to many sophisticated techniques, such as data augmentation, pre-training, knowledge distillation, and multilingual modeling. We analyze and summarize the application issues, which include real-time, segmentation, named entity, gender bias, and code-switching. Finally, we discuss some promising directions for future work.

How to learn a better speech representation for end-to-end speech-to-text translation (ST) with limited labeled data?Existing techniques often attempt to transfer powerful machine translation (MT) capabilities to ST, but neglect the representation discrepancy across modalities.In this paper, we propose the Speech-TExt Manifold Mixup (STEMM) method to calibrate such discrepancy.Specifically, we mix up the representation sequences of different modalities, and take both unimodal speech sequences and multimodal mixed sequences as input to the translation model in parallel, and regularize their output predictions with a selflearning framework.Experiments on MuST-C speech translation benchmark and further analysis show that our method effectively alleviates the cross-modal representation discrepancy, and achieves significant improvements over a strong baseline on eight translation directions.* indicates corresponding authors.

How can we learn unified representations for spoken utterances and their written text?Learning similar representations for semantically similar speech and text is important for speech translation.To this end, we propose ConST, a cross-modal contrastive learning method for end-to-end speech-to-text translation.We evaluate ConST and a variety of previous baselines on a popular benchmark MuST-C.Experiments show that the proposed ConST consistently outperforms the previous methods, and achieves an average BLEU of 29.4.The analysis further verifies that ConST indeed closes the representation gap of different modalities -its learned representation improves the accuracy of cross-modal speechtext retrieval from 4% to 88%.Code and models are available at https://github. com/ReneeYe/ConST.

Apathy is a debilitating feature of many neuropsychiatric diseases, that is typically described as a reduction of goal-directed behaviour. Despite its prevalence and prognostic importance, the mechanisms underlying apathy remain controversial. Degeneration of the locus coeruleus-noradrenaline system is known to contribute to motivational deficits, including apathy. In healthy people, noradrenaline has been implicated in signalling the uncertainty of expectations about the environment. We proposed that noradrenergic deficits contribute to apathy by modulating the relative weighting of prior beliefs about action outcomes. We tested this hypothesis in the clinical context of Parkinson's disease, given its associations with apathy and noradrenergic dysfunction. Participants with mild-to-moderate Parkinson's disease (N = 17) completed a randomised double-blind, placebo-controlled, crossover study with 40 mg of the noradrenaline reuptake inhibitor atomoxetine. Prior weighting was inferred from psychophysical analysis of performance in an effort-based visuomotor task, and was confirmed as negatively correlated with apathy. Locus coeruleus integrity was assessed in vivo using magnetisation transfer imaging at ultra-high field 7T. The effect of atomoxetine depended on locus coeruleus integrity: participants with a more degenerate locus coeruleus showed a greater increase in prior weighting on atomoxetine versus placebo. The results indicate a contribution of the noradrenergic system to apathy and potential benefit from noradrenergic treatment of people with Parkinson's disease, subject to stratification according to locus coeruleus integrity. More broadly, these results reconcile emerging predictive processing accounts of the role of noradrenaline in goal-directed behaviour with the clinical symptom of apathy and its potential pharmacological treatment.

An end-to-end speech-to-text translation (ST) takes audio in a source language and outputs the text in a target language. Existing methods are limited by the amount of parallel corpus. Can we build a system to fully utilize signals in a parallel ST corpus? We are inspired by human understanding system which is composed of auditory perception and cognitive processing. In this paper, we propose Listen-Understand-Translate, (LUT), a unified framework with triple supervision signals to decouple the end-to-end speech-to-text translation task. LUT is able to guide the acoustic encoder to extract as much information from the auditory input. In addition, LUT utilizes a pre-trained BERT model to enforce the upper encoder to produce as much semantic information as possible, without extra data. We perform experiments on a diverse set of speech translation benchmarks, including Librispeech English-French, IWSLT English-German and TED English-Chinese. Our results demonstrate LUT achieves the state-of-the-art performance, outperforming previous methods. The code is available at https://github.com/dqqcasia/st.

End-to-end speech translation models have become a new trend in research due to their potential of reducing error propagation.However, these models still suffer from the challenge of data scarcity.How to effectively use unlabeled or other parallel corpora from machine translation is promising but still an open problem.In this paper, we propose Cross Speech-Text Network (XSTNet), an end-to-end model for speech-to-text translation.XSTNet takes both speech and text as input and outputs both transcription and translation text.The model benefits from its three key design aspects: a self-supervised pretrained sub-network as the audio encoder, a multi-task training objective to exploit additional parallel bilingual text, and a progressive training procedure.We evaluate the performance of XSTNet and baselines on the MuST-C En-X and LibriSpeech En-Fr datasets.In particular, XSTNet achieves state-of-the-art results on all language directions with an average BLEU of 28.8, outperforming the previous best method by 3.2 BLEU.Code, models, cases, and more detailed analysis are available at https://github.com/ReneeYe/XSTNet.

Approximate computing, being able to tradeoff computation quality (e.g., accuracy) and computational effort (e.g., energy) for error-tolerant applications such as media processing and the emerging Recognition, Mining, and Synthesis (RMS) applications, has gained significant traction in recent years. Many of these applications employ iterative methods for solution-finding, wherein a sequence of improving approximate solutions are generated before reaching the final converged solution. In this work, we propose ApproxIt, a novel approximate computing framework for iterative methods with quality guarantees. To be specific, we present a lightweight quality estimator that is able to capture the solution quality of each iteration and use it to guide the selection of approximate computing mode in the next iteration. With the proposed dynamic effort scaling technique, ApproxIt is able to dramatically improve application energy efficiency under quality guarantees, as demonstrated in our experimental results.

Learning-based dynamic power management (DPM) techniques, being able to adapt to varying system conditions and workloads, have attracted a lot of research attention recently. To the best of our knowledge, however, none of the existing learning-based DPM solutions are dedicated to power reduction in multicore processors, although they can be utilized by treating each processor core as a standalone entity and conducting DPM for them separately. In this paper, by including task allocation into our learning-based DPM framework for multicore processors, we are able to manipulate idle periods on processor cores to achieve a better tradeoff between power consumption and system performance. Experimental results show that the proposed solution significantly outperforms existing DPM techniques.

Region-based multi-supply voltage (MSV) design, by which circuits are partitioned into multiple "voltage islands" and each island operates at a supply voltage that meets its own performance requirement, is an effective technique to tradeoff power and performance. Different from conventional voltage island generation techniques that work in a conservative manner to guarantee "always correct" computation, in this work, we investigate the MSV design problem for timing-speculative circuits, which achieves high energy-efficiency by allowing the occurrence of infrequent timing errors and correcting them online. A novel algorithm based on dynamic programming is developed to tackle this problem. Experimental results on various benchmark circuits demonstrate the effectiveness of the proposed methodology.

Integrated circuits suffer from severe variation effects with technology scaling, making their timing behavior increasingly unpredictable. Timing speculation is a promising technique to tackle this problem with the help of online timing error detection and correction mechanisms. In this paper, we propose to use redundancy addition and removal (RAR) technique to optimize timing-speculated circuits. By intentionally removing wires on those frequently-exercised critical paths and replacing them with wires on less critical ones (if possible), the proposed technique is able to greatly reduce the timing error rate of the circuit and improve its overall throughput, as shown in our experimental results on various benchmark circuits.

Approximate circuit designs allow us to tradeoff computation quality (e.g., accuracy) and computational effort (e.g., energy), by exploiting the inherent error-resilience of many applications. As the computation quality requirement of an application generally varies at runtime, it is preferable to be able to reconfigure approximate circuits to satisfy such needs and save unnecessary computational effort. In this paper, we present a reconfiguration-oriented design methodology for approximate circuits, and propose a reconfigurable approximate adder design that degrades computation quality gracefully. The proposed design methodology enables us to achieve better quality-effort tradeoff when compared to existing techniques, as demonstrated in the application of DCT computing.

With technology scaling, integrated circuits suffer from increasingly severe static and dynamic variations, which often manifest themselves as infrequent timing errors on circuit speed paths, if a large timing guard-band is not reserved. This paper presents a new forward timing error correction scheme, namely ForTER, which predicts whether the occurrence of timing errors would propagate to the next level of sequential elements and corrects them without necessarily borrowing timing slack. The proposed technique can be combined with other timing error resilient circuit design techniques to further improve circuit performance, as demonstrated in our experimental results with various benchmark circuits.

Learning-based dynamic power management (DPM) techniques, being able to adapt to varying system conditions and workloads, have attracted lots of research attention recently. To the best of our knowledge, however, none of the existing learning-based DPM solutions are dedicated to power reduction in multi-core processors, although they can be utilized by treating each processor core as a standalone entity and conducting DPM for them separately. In this work, by including task allocation into our learning-based DPM framework for multi-core processors, we are able to manipulate idle periods on processor cores to achieve a better tradeoff between power consumption and system performance. Experimental results show that the proposed solution significantly outperforms existing DPM techniques.

By assigning intentional clock arrival times to the sequential elements in a circuit, clock skew scheduling (CSS) techniques can be utilized to improve IC performance. Existing CSS solutions work in a conservative manner that guarantees “always correct” computation, and hence their effectiveness is greatly challenged by the ever-increasing process variation effects. By allowing infrequent timing errors and recovering from them with minor performance impact, timing speculation techniques such as Razor have gained wide interests from both academia and industry. In this work, we formulate the clock skew scheduling problem for circuits equipped with timing speculation capability and propose a novel CSS algorithm based on gradient-descent method. Experimental results on various benchmark circuits demonstrate the effectiveness of our proposed methodology.

By allowing the occurrence of infrequent timing errors and correcting them with rollback mechanisms, the so-called timing speculation (TS) technique can significantly improve circuit energy-efficiency and hence has become one of the most promising solutions to mitigate the ever-increasing variation effects in nanometer technologies. As timing error recovery incurs non-trivial performance/energy overhead, it is important to reshape the delay distribution of critical paths in timing-speculated circuits to minimize their timing error rates. Most existing TS optimization techniques achieve this objective with post-synthesis techniques such as gate sizing or body biasing. In this work, we propose to conduct logic synthesis for timing-speculated circuits from the ground up. Being able to manipulate circuit structures during logic optimization, the proposed solution is able to dramatically reduce circuit timing error rates and hence improve its throughput, as demonstrated with experimental results on various benchmark circuits.

Today's multiprocessor system-on-a-chip (MPSoC) products typically have multiple execution modes, and for each mode, all the products utilize the same task allocation and schedule strategy determined at design stage. As these products experience different usages by customers, such unified solution can at best be optimized for a hypothetical common case. It is hence likely that the product is not reliable or energy-efficient from particular customers' point of view. To tackle this problem, we propose a novel customer-aware task allocation and scheduling technique, wherein we generate an initial task schedule for each execution mode at design stage and then perform online adjustment at regular intervals for lifetime reliability improvement and/or energy reduction according to the specific usage strategy of individual products. Experimental results on several hypothetical MPSoCs with various task graphs demonstrate the effectiveness of the proposed personalized solution.

The timing performance and yield of integrated circuits can be improved by carefully assigning intentional clock skews to flip-flops. Due to the ever-increasing process, voltage, and temperature variations with technology scaling, however, traditional clock skew optimization solutions that work in a conservative manner to guarantee “always correct” computation cannot perform as well as expected. By allowing infrequent timing errors and recovering from them with minor performance impact, the concept of timing speculation has attracted lots of research attention since it enables “better than worst-case design”. In this work, we propose a novel online clock skew tuning technique for circuits equipped with timing speculation capability. By observing the occurrence of timing errors at runtime and tuning clock skews accordingly, the proposed technique is able to achieve much better timing performance when compared to existing clock skew optimization solutions. Experimental results on various benchmark circuits demonstrate the effectiveness of the proposed methodology.

Three-dimensional (3D) memory products are emerging to fulfill the ever-increasing demands of storage capacity. In 3D-stacked memory, redundancy sharing between neighboring vertical memory blocks using short through-silicon vias (TSVs) is a promising solution for yield enhancement. Since different memory dies are with distinct fault bitmaps, how to selectively matching them together to maximize the yield for the bonded 3D-stacked memory is an interesting and relevant problem. In this paper, we present novel solutions to tackle the above problem. Experimental results show that the proposed methodology can significantly increase memory yield when compared to the case that we only bond self-reparable dies together.