Attributed text generation aims to enhance the reliability of content generated from large language models by providing citations for each claim, which thereby enables users to easily verify the correctness of the responses.However, the scarcity of high-quality training samples presents a significant challenge in aligning large language models to generate texts with citations, revealing considerable room for improvement in existing attribution systems.Besides, existing approaches of aligning large language models to follow user instructions can lead to an undue emphasis on irrelevant documents, which in turn reduces the quality of responses.To address the above problems, we propose Automatic Alignment Framework for Attributed Text Generation (A 3 ), a novel framework designed to automatically generate highquality attributed query-response pairs for both supervised fine-tuning and preference optimization stages without human annotation.With the help of A 3 , Mistral-7B can achieve a citation recall of 84.4 and a precision of 87.0 precision on ASQA, which notably surpasses GPT-4's citation recall of 73.0 and precision of 76.5. 1

Generative Reward Models (GenRMs) leverage synthesized Chains of Thought (CoT) to reduce the need for massive labeled data, but this approach introduces risks of overoptimization due to the inability to guarantee the correctness of the CoTs.Identifying and optimizing unexpected behaviors within these synthesized CoT remains a challenge, as it heavily depends on precise annotations of intermediate behavior, similar to process supervision.In this work, we introduce a criteria-based preference tree for reward modeling, where each path in the tree represents a reasoning trajectory based on synthesized criteria.Crucially, each reasoning trajectory can be independently optimized through RL algorithm.These fine-grained process reward signals are derived from the inferencetime computations and predefined rules, eliminating the need for human supervision.In experiments, SyncPL 1 showed significant improvements over baselines on multiple human preference benchmarks.We further demonstrate that synthesized data can be learned using a long CoT format, analogous to an o1-like model, further enhancing performance while keeping stability and efficiency during training.

In this paper, we propose a method for generating environment maps for sky illumination by using an atmospheric radiative transfer simulator, AT3D, with meteorological satellite data used for cloud modeling as volumetric grid data and previously measured concentrations of atmospheric molecular and aerosols. We present generated sky environment maps, their respective luminance intensity histograms, and images rendered using the maps as sky illumination and compare them with those of captured spherical images.

Skill-based deep reinforcement learning (DRL) typically embeds skills extracted from expert demonstrations into a large latent space. However, it is difficult for agents to explore the latent skill space, because only a small portion of the space is relevant to a given robot state. To this end, we propose a three-layer algorithm to realize that the agent can adaptively select skills and directly explore the skill region associated with the state. The high-level policy outputs skill categories with a given state; then, a conditional probability model is constructed using the state and skill categories to guide the agent’s exploration. We first learn high- and mid-level skill prior models and a low-level controller with offline data. Next, we learn high-level and mid-level policies to construct two different action spaces. The high-level action is used to guide the mid-level policy to explore skill-related regions; the mid-level is used to guide low-level models to generate skill-related robot actions. Finally, we validate our approach in three challenging manipulation tasks, and the experimental results show that our approach can significantly speed up exploration and cross-task learning.

Imbalanced data always has a serious impact on a predictive model, and most under-sampling techniques consume more time and suffer from loss of samples containing critical information during imbalanced data processing, especially in the biomedical field. To solve these problems, we developed an active balancing mechanism (ABM) based on valuable information contained in the biomedical data. ABM adopts the Gaussian naïve Bayes method to estimate the object samples and entropy as a query function to evaluate sample information and only retains valuable samples of the majority class to achieve under-sampling. The Physikalisch Technische Bundesanstalt diagnostic electrocardiogram (ECG) database, including 5,173 normal ECG samples and 26,654 myocardial infarction ECG samples, is applied to verify the validity of ABM. At imbalance rates of 13 and 5, experimental results reveal that ABM takes 7.7 seconds and 13.2 seconds, respectively. Both results are significantly faster than five conventional under-sampling methods. In addition, at the imbalance rate of 13, ABM-based data obtained the highest accuracy of 92.23% and 97.52% using support vector machines and modified convolutional neural networks (MCNNs) with eight layers, respectively. At the imbalance rate of 5, the processed data by ABM also achieved the best accuracy of 92.31% and 98.46% based on support vector machines and MCNNs, respectively. Furthermore, ABM has better performance than two compared methods in F 1-measure, G-means, and area under the curve. Consequently, ABM could be a useful and effective approach to deal with imbalanced data in general, particularly biomedical myocardial infarction ECG datasets, and the MCNN can also achieve higher performance compared to the state of the art.