New NLP benchmarks are urgently needed to align with the rapid development of large language models (LLMs). We present a meticulously designed evaluation benchmark that leverages the knowledge graph. This evaluation comprises 584 level-1 knowledge points and 1,989 level-2 knowledge points, thereby encompassing a comprehensive spectrum of the K12 education domain knowledge. The primary objective is to comprehensively assess the high-level comprehension aptitude and reasoning capabilities of LLMs operating within the Chinese context. Our evaluation incorporates five distinct question types with 39,452 questions. We test the current mainstream LLMs by three distinct modes. Firstly, four prompt evaluation modes were employed to assess the fundamental capacity. Additionally, for choice questions, a result-oriented evaluation approach was designed through data augmentation to assess the model's proficiency in advanced knowledge and reasoning. Moreover, a subset with reasoning process is derived, and the process-oriented testing method is used to test the model's interpretability and higher-order reasoning capacity. We further show models' capability in our knowledge points, and anticipate the evaluation can assist in the assessment of the strengths and deficiencies of LLMs on knowledge points, thus fostering their development within the Chinese context. Our Dataset will be publicly available in https://github.com/tal-tech/chinese-k12-evaluation.

The tool-use Large Language Models (LLMs) that integrate with external Python interpreters have significantly enhanced mathematical reasoning capabilities for open-source LLMs, while tool-free methods chose another track: augmenting math reasoning data.However, a great method to integrate the above two research paths and combine their advantages remains to be explored.In this work, we firstly include new math questions via multi-perspective data augmenting methods and then synthesize code-nested solutions to them.The open LLMs (e.g., Llama-2) are finetuned on the augmented dataset to get the resulting models, MuMath-Code (µ-Math-Code).During the inference phase, our MuMath-Code generates code and interacts with the external python interpreter to get the execution results.Therefore, MuMath-Code leverages the advantages of both the external tool and data augmentation.To fully leverage the advantages of our augmented data, we propose a two-stage training strategy: In Stage-1, we finetune Llama-2 on pure CoT data to get an intermediate model, which then is trained on the code-nested data in Stage-2 to get the resulting MuMath-Code.Our MuMath-Code-7B achieves 83.8% on GSM8K and 52.4% on MATH, while MuMath-Code-70B model achieves new state-of-the-art performance among open methods-achieving 90.7% on GSM8K and 55.1% on MATH.Extensive experiments validate the combination of tool use and data augmentation, as well as our two-stage training strategy.We release the proposed dataset along with the associated code for public use: https://github.com/ youweihao-tal/MuMath-Code.

In the current state of 3D human pose and shape estimation, the two-stage methods involving 3D joints estimation and inverse kinematics (IK) to obtain more accurate human pose, have gained significant traction. However, the effectiveness of the final optimized depth is greatly influenced by the initial 3D joints estimation. The 3D joints errors primarily caused by depth estimation. Our findings indicate that 2D observations (2D joint locations and Confidence) offer valuable supplementary data. Depth estimation errors are greater for low-confidence joints, but high-confidence joints yield more accuracy. In light of this, we propose a Biased Depth Propagation (BDP) module that leverages the confidence of 2D joints to enhance depth estimation. This module enables us to improve the depth estimation of low-confidence joints by propagating the depth information from high-confidence joints. Furthermore, small variations in the SMPL parameters can significantly impact the reconstructed meshes causing misalignment with image evidence. Therefore, we propose the Refined Joint Aligned (RJA) module to fine-tune the SMPL parameters by utilizing enhanced joint ROI features captured by the estimated 2D joint locations. Finally, we have validated the performance of our method on popular datasets such as Human3.6m and 3DPW for human mesh recovery task, and achieved SOTA results, demonstrate the effectiveness of our Depth Propagation and Alignment with 2D Observations Guidance(DPA-2D).