Yiyao Yu, Yuxiang Zhang, Dongdong Zhang, Xiao Liang, Hengyuan Zhang, Xingxing Zhang, Mahmoud Khademi, Hany Hassan Awadalla, Junjie Wang, Yujiu Yang, Furu Wei. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Hengyuan Zhang, Chenming Shang, Sizhe Wang, Dongdong Zhang, Yiyao Yu, Feng Yao, Renliang Sun, Yujiu Yang, Furu Wei. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Power load forecasting is a key link in the secondary distribution of grid energy and stable system operation. Traditional point forecasting methods only provide a single prediction value, which makes it difficult to cope with the randomness and volatility of load demand and cannot meet the demand for accurate scheduling of power systems. For this reason, this paper proposes a TCN-BiLSTM-Attention model based on Differentiated Creative Search (DCS) optimization for power load quartile interval forecasting. The model extracts the time series features by Temporal Convolutional Network (TCN), captures the before-and-after dependencies of the sequences using Bidirectional Long Short-term Memory (BiLSTM), and introduces the Attention mechanism (AM) to enhance the focus on key features. The DCS algorithm is used to optimize the hyperparameters to ensure model performance. Based on point prediction, the model provides confidence interval prediction, which enhances the reliability of the results and decision support capability. Comparative experiments show that the model quantifies the forecast uncertainty while improving the load forecasting accuracy, which helps decision-makers develop corresponding operation plans based on different risk preferences.

A unified foundation model for medical time series—pretrained on open access and ethically reviewed medical corpora—offers the potential to reduce annotation burdens, minimize model customization, and enable robust transfer across clinical institutions, modalities, and tasks, particularly in data-scarce or privacy-constrained environments. However, existing time series foundation models struggle to handle medical time series data due to its inherent challenges, including irregular intervals, heterogeneous sampling rates, and frequent missingness. To address these challenges, we introduce MIRA, a unified foundation model specifically designed for medical time series forecasting. MIRA incorporates a Continuous-Time Rotary Positional Encoding that enables fine-grained modeling of variable time intervals, a frequency-specific mixture-of-experts layer that routes computation across latent frequency regimes to further promote temporal specialization, and a Continuous Dynamics Extrapolation Block based on Neural ODE that models the continuous trajectory of latent states, enabling accurate forecasting at arbitrary target timestamps. Pretrained on a large-scale and diverse medical corpus comprising over 454 billion time points collect from publicly available datasets, MIRA achieving reductions in forecasting errors by an average of 8% and 6% in out-of-distribution and in-distribution scenarios, respectively. We also introduce a comprehensive benchmark spanning multiple downstream clinical tasks, establishing a foundation for future research in medical time series modeling.

Process Reward Models (PRMs) have shown promise in enhancing the mathematical reasoning capabilities of Large Language Models (LLMs) through Test-Time Scaling (TTS). However, their integration into multimodal reasoning remains largely unexplored. In this work, we take the first step toward unlocking the potential of PRMs in multimodal mathematical reasoning. We identify three key challenges: (i) the scarcity of high-quality reasoning data constrains the capabilities of foundation Multimodal Large Language Models (MLLMs), which imposes further limitations on the upper bounds of TTS and reinforcement learning (RL); (ii) a lack of automated methods for process labeling within multimodal contexts persists; (iii) the employment of process rewards in unimodal RL faces issues like reward hacking, which may extend to multimodal scenarios. To address these issues, we introduce URSA, a three-stage Unfolding multimodal pRocess-Supervision Aided training framework. We first construct MMathCoT-1M, a high-quality large-scale multimodal Chain-of-Thought (CoT) reasoning dataset, to build a stronger math reasoning foundation MLLM, URSA-8B. Subsequently, we go through an automatic process to synthesize process supervision data, which emphasizes both logical correctness and perceptual consistency. We introduce DualMath-1.1M to facilitate the training of URSA-8B-RM. Finally, we propose Process-Supervised Group-Relative-Policy-Optimization (PS-GRPO), pioneering a multimodal PRM-aided online RL method that outperforms vanilla GRPO. With PS-GRPO application, URSA-8B-PS-GRPO outperforms Gemma3-12B and GPT-4o by 8.4% and 2.7% on average across 6 benchmarks.

Antimicrobial peptides (AMPs) are vital components of the natural immune system. With their broad-spectrum antibacterial properties, low resistance, and multifunctionality, they are considered important molecules for addressing the problem of bacterial resistance. At the same time, with the rapid development of large language models, their powerful intelligence has made natural language an increasingly important bridge for human-machine interaction, bringing new perspectives to AMP design. By integrating natural language with protein language, we can significantly reduce the complexity of tasks, enhance intuitiveness and operational convenience, and deeply incorporate expert knowledge, thus providing new insights for exploring and understanding peptide design. Based on these benefits, we propose Text-to-AMP, an innovative framework that uses natural language descriptions to directly guide AMP sequence generation. Through the integration of multimodal implicit alignment, explicit alignment, and autoregressive sequence generation, Text-to-AMP establishes a semantic connection between natural language and peptide sequences, enabling precise and interpretable peptide design. Comprehensive experimental evaluations demonstrate that our model outperforms baseline methods on most metrics in AMP activity prediction tasks. In zero-shot text-guided sequence generation tasks, the model surpasses benchmarks, including the GPT series, by generating novel peptide sequences with enhanced antimicrobial activity and reduced toxicity. Furthermore, case studies validate the model’s capability to meet fine-grained structural and functional design criteria. In conclusion, Text-to-AMP introduces a advanced paradigm for AMP design by significantly lowering technical barriers and accelerating the discovery of antimicrobial peptides.