Diffusion models have shown impressive performance in capturing complex and multi-modal action distributions for game agents, but their slow inference speed prevents practical deployment in real-time game environments. While consistency models offer a promising approach for one-step generation, they often suffer from training instability and performance degradation when applied to policy learning. In this paper, we present CPQE (Consistency Policy with Q-Ensembles), which combines consistency models with Q-ensembles to address these challenges. CPQE leverages uncertainty estimation through Q-ensembles to provide more reliable value function approximations, resulting in better training stability and improved performance compared to classic double Q-network methods. Our extensive experiments across multiple game scenarios demonstrate that CPQE achieves inference speeds of up to 60 Hz - a significant improvement over state-of-the-art diffusion policies that operate at only 20 Hz - while maintaining comparable performance to multi-step diffusion approaches. CPQE consistently outperforms state-of-theart consistency model approaches, showing both higher rewards and enhanced training stability throughout the learning process. These results indicate that CPQE offers a practical solution for deploying diffusion-based policies in games and other real-time applications where both multi-modal behavior modeling and rapid inference are critical requirements.

Diffusion policy has shown a strong ability to express complex action distributions in offline reinforcement learning (RL). However, it suffers from overestimating Q-value functions on out-of-distribution (OOD) data points due to the offline dataset limitation. To address it, this paper proposes a novel entropy-regularized diffusion policy and takes into account the confidence of the Q-value prediction with Q-ensembles. At the core of our diffusion policy is a mean-reverting stochastic differential equation (SDE) that transfers the action distribution into a standard Gaussian form and then samples actions conditioned on the environment state with a corresponding reverse-time process. We show that the entropy of such a policy is tractable and that can be used to increase the exploration of OOD samples in offline RL training. Moreover, we propose using the lower confidence bound of Q-ensembles for pessimistic Q-value function estimation. The proposed approach demonstrates state-of-the-art performance across a range of tasks in the D4RL benchmarks, significantly improving upon existing diffusion-based policies. The code is available at https://github.com/ruoqizzz/entropy-offlineRL.

This paper focuses on the challenge of surface defect detection in manufacturing, particularly under conditions of background variation and noise interference. To tackle this issue, a novel Background-Adaptive Surface Defect Detection Network (BANet) is proposed. The BANet enhances the defect detection capabilities by improving generalization capacity through learning comparative abilities between positive samples and testing samples. In order to mitigate the impact of three types of noise (texture variation, translation, and rotation), a Foreground Edge Attention Mechanism (FEAM) and a Spatial Transformer Module (STM) are introduced. The FEAM enhances the model's ability to differentiate between foreground and background, thereby effectively reducing texture variation noise. The STM uses affine transformations to eliminate translation and rotation noise. The effectiveness of the proposed network is validated on Optical Communication Devices (OCDs) dataset, with results indicating superior performance compared to prevailing state-of-the-art methods. The findings of this study highlight the potential of our approach in effectively addressing surface defect detection in variable backgrounds and noisy conditions, thereby contributing significantly to the quality and reliability of manufacturing processes.