Recent advances in reasoning models have demonstrated significant improvements in accuracy by employing detailed and comprehensive reasoning processes.However, generating these lengthy reasoning sequences is computationally expensive and time-consuming.To address this inefficiency, we leverage the inherent parallelizability of certain tasks to accelerate the reasoning process.Specifically, when multiple parallel reasoning steps exist, we decode multiple tokens per forward pass via a tree-like attention mask within a single sequence, avoiding additional memory usage.Experimental results show that our method achieves up to nearly 100% speedup in decoding while basically maintaining the answer quality.Our code is available in https://github.com/yuyijiong/parallel- decoding-in-one-sequence.

Large language models have shown considerable potential in areas such as program writing and API invocation, significantly enhancing automation and efficiency. However, frequent updates to APIs pose significant challenges to these models, particularly in terms of the high costs of retraining and the complexity of maintenance associated with updating the model’s knowledge. Current strategies for handling these issues generally fall into two categories: enhanced reasoning and parameter tuning. Among these, Retrieval Augmented Generation (RAG), as a primary method for enhancing reasoning, assists the code models in the reasoning process during code generation by modifying prompt information. However, since it does not alter the model’s parameters, external knowledge may conflict with the knowledge stored in the model after training; On the other hand, the method of parameter tuning incorporates API updates into the training data, modifying the code model’s parameters and enabling re-reasoning. However it struggles due to the scarcity of high-quality samples for effective model training. In response to these challenges, we propose an innovative learning method that utilizes the logical reasoning capabilities of generic large models and adopts a teacher-student framework. Within our framework, the teacher model analyzes the historical learning records of the code model to autonomously generate high-quality programming problems and their solutions, guiding the model through preference-based training. This approach not only leverages the strengths of large models in problem generation but also strictly controls the quality of learning materials, effectively avoiding the knowledge conflicts that can arise from RAG-based methods. Our Method offers a more effective and practical solution for code models in rapidly evolving API environments, significantly improving their stability and accuracy in practical applications. Experiments conducted across multiple common code repositories have demonstrated the significant advantages of our method, confirming its proactive and practical capabilities in addressing the challenges of API updates.

The overflowing of textual data on the web needs an efficient tool that is able to manage and process data. In this context, automatic text summarization has shown a great importance in several application areas. It aims to create a coherent and fluent short version of a document while preserving of the main information. This method allows for a reduction in reading time by condensing relevant information from a large collection of documents. Several automatic text summarization approaches have been proposed in order to entail shorten parts of the document. These methods have good results, but they still need improvements related to the reliability of sentences extraction, redundancy, semantic relationships between sentences, etc. This paper introduces a new hybrid architecture, combining a 2-layer recurrent neural network (RNN) extractive model and a sequence-to-sequence attentional abstractive model. This method uses the advantages of both extractive and abstractive approaches. A given text is first fed into the extractive model to obtain its relevant content, then generalized using the abstractive model, resulting in the text's summary. First experimental results on real-world data show that the proposed model can achieve competitive results for extractive, abstractive and hybrid models.