The pursuit of real-time agentic interaction has driven interest in Diffusion-based Large Language Models (dLLMs) as alternatives to autoregressive backbones, promising to break the sequential latency bottleneck.However, does such efficiency gains translate into effective agentic behavior?In this work, we present a comprehensive evaluation of dLLMs (e.g., LLaDA, Dream) across two distinct agentic paradigms: Embodied Agents (requiring longhorizon planning) and Tool-Calling Agents (requiring precise formatting).Contrary to the efficiency hype, our results on Agentboard and BFCL reveal a "bitter lesson": current dLLMs fail to serve as reliable agentic backbones, frequently leading to systematic failure.(1) In Embodied settings, dLLMs suffer repeated attempts, failing to branch under temporal feedback.(2) In Tool-Calling settings, dLLMs fail to maintain symbolic precision (e.g.strict JSON schemas) under diffusion noise.To assess the potential of dLLMs in agentic workflows, we introduce DiffuAgent, a multi-agent evaluation framework that integrates dLLMs as plug-and-play cognitive cores.Our analysis shows that dLLMs are effective in non-causal roles (e.g., memory summarization and tool selection) but require the incorporation of causal, precise, and logically grounded reasoning mechanisms into the denoising process to be viable for agentic tasks.

Large Language Models (LLMs) have shown significant potential as judges for Machine Translation (MT) quality assessment, providing both scores and fine-grained feedback. Although approaches such as GEMBA-MQM have shown state-of-the-art performance on reference-free evaluation, the predicted errors do not align well with those annotated by human, limiting their interpretability as feedback signals. To enhance the quality of error annotations predicted by LLM evaluators, we introduce a universal and training-free framework, MQM-APE, based on the idea of filtering out non-impactful errors by Automatically Post-Editing (APE) the original translation based on each error, leaving only those errors that contribute to quality improvement. Specifically, we prompt the LLM to act as 1) evaluator to provide error annotations, 2) post-editor to determine whether errors impact quality improvement and 3) pairwise quality verifier as the error filter. Experiments show that our approach consistently improves both the reliability and quality of error spans against GEMBA-MQM, across eight LLMs in both high- and low-resource languages. Orthogonal to trained approaches, MQM-APE complements translation-specific evaluators such as Tower, highlighting its broad applicability. Further analysis confirms the effectiveness of each module and offers valuable insights into evaluator design and LLMs selection.

The pretrained language model (PLM) based metrics have been successfully used in evaluating language generation tasks.Recent studies of the human evaluation community show that considering both major errors (e.g.mistranslated tokens) and minor errors (e.g.imperfections in fluency) can produce high-quality judgments.This inspires us to approach the final goal of the automatic metrics (human-like evaluations) by fine-grained error analysis.In this paper, we argue that the ability to estimate sentence confidence is the tip of the iceberg for PLM-based metrics.And it can be used to refine the generated sentence toward higher confidence and more reference-grounded, where the costs of refining and approaching reference are used to determine the major and minor errors, respectively.To this end, we take BARTScore as the testbed and present an innovative solution to marry the unexploited sentence refining capacity of BARTScore and human-like error analysis, where the final score consists of both the evaluations of major and minor errors.Experiments show that our solution consistently improves BARTScore, outperforming top-scoring metrics in 19/25 test settings.Analyses demonstrate our method robustly and efficiently approaches human-like evaluations, enjoying better interpretability.Our code and scripts will be publicly released in https: //github.com/Coldmist-Lu/ ErrorAnalysis_NLGEvaluation.