Xin Chen, Feng Jiang, Yiqian Zhang, Hardy Chen, Shuo Yan, Wenya Xie, Min Yang, Shujian Huang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Process-supervised reward models serve as a fine-grained function that provides detailed step-wise feedback to model responses, facilitating effective selection of reasoning trajectories for complex tasks.Despite its advantages, evaluation on PRMs remains less explored, especially in the multimodal domain.To address this gap, this paper first benchmarks current vision large language models (VLLMs) as two types of reward models: output reward models (ORMs) and process reward models (PRMs) on multiple vision-language benchmarks, which reveal that neither ORM nor PRM consistently outperforms across all tasks, and superior VLLMs do not necessarily yield better rewarding performance.To further advance evaluation, we introduce VILBENCH, a vision-language benchmark designed to require intensive process reward signals.Notably, Ope-nAI's GPT-4o with Chain-of-Thought (CoT) achieves only 27.3% accuracy, challenging current VLLMs.Lastly, we preliminarily showcase a promising pathway towards bridging the gap between general VLLMs and reward models-by collecting 73.6K vision-language process reward data using an enhanced treesearch algorithm, our 3B model is able to achieve an average improvement of 3.3% over standard CoT and up to 2.5% compared to its untrained counterpart on VILBENCH by selecting OpenAI o1's generations.We will release our code, model, and data at https: //ucsc-vlaa.github.io/ViLBench.

Wentao Ge, Shunian Chen, Hardy Chen, Nuo Chen, Junying Chen, Zhihong Chen, Wenya Xie, Shuo Yan, Chenghao Zhu, Ziyue Lin, Dingjie Song, Xidong Wang, Anningzhe Gao, Zhang Zhiyi, Jianquan Li, Xiang Wan, Benyou Wang. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.