Non-Markovian Tasks (NMTs) are distinguished by their dependence on long-term memory and state-dependent dynamics, setting them apart from the traditional Markovian models typically employed in Reinforcement Learning (RL). NMTs not only suffer from reward sparseness but also rely on historical information, making their resolution considerably more challenging. In this paper, we propose a novel RL framework T4NMTD (Transition-centric framework for NMT Decomposition), designed specifically for learning NMTs which are specified by temporal logic. The core of T4NMTD is a task decomposition mechanism along with a parallel training approach for NMTs. An NMT is first decomposed as basic units based on the transitions of the automata which are derived from temporal logic formulae. The units are then modularized into sub-tasks according to their semantic similarity under logical interpretation. The training strategy of T4NMTD adopts a dual-level structure: the high-level learns to shape the boundaries and coordinate arrangement of the sub-tasks from a global perspective, while the low-level learns those sub-tasks in parallel. In addition, we invent a dynamic policy intervention scheme to mitigate the policy myopic issue during parallel training. A comprehensive evaluation is conducted on benchmark problems with respect to various metrics. The experimental results demonstrate that T4NMTD effectively addresses NMTs, achieving significant performance improvements compared with related studies.

Automated code generation has long been considered the holy grail of software engineering. The emergence of Large Language Models (LLMs) has catalyzed a revolutionary breakthrough in this area. However, existing methods that only rely on LLMs remain inadequate in the quality of generated code, offering no guarantees of satisfying practical requirements. They lack a systematic strategy for requirements development and modeling. Recently, LLM-based agents typically possess powerful abilities and play an essential role in facilitating the alignment of LLM outputs with user requirements. In this paper, we envision the first multi-agent framework for reliable code generation based on Requirements Development and Formalization, named ReDeFo. This framework incorporates three agents, highlighting their augmentation with knowledge and techniques of formal methods, into the requirements-to-code generation pipeline to strengthen quality assurance. The core of ReDeFo is the use of formal specifications to bridge the gap between potentially ambiguous natural language requirements and precise executable code. ReDeFo enables rigorous reasoning about correctness, uncovering hidden bugs, and enforcing critical properties throughout the development process.

Interrupt-driven programs are widely used in safety-critical fields like aerospace and embedded systems. However, the unpredictable interleaving of Interrupt Service Routines (ISRs) can lead to concurrency bugs, particularly atomicity violations when ISRs preempt atomic sequences of instructions. To address this, we propose a dynamic approach for detecting atomicity violations in interrupt-driven programs. Extensive experiments demonstrate that our method is more precise and efficient than related approaches.

Certificate parsers, which are critical components of Secure Sockets Layer or Transport Layer Security (SSL/TLS) implementations, parse incomprehensible certificates into comprehensible inputs to certificate validators and humans. Thus, certificate parsers profoundly affect decision-makings of validators and humans, which in turn affect security. To guarantee the correctness of certificate parsers, an approach for search-based differential testing of certificate parsers, namely SBDT, is put forward. SBDT begins with modeling certificate structures, mutation operations, and bounds. Based on the initial model, SBDT searches for the most promising model node and mutation operator that trigger discrepancies, and generates a certificate from the node and operator it finds. Then, SBDT feeds the certificate to certificate parsers, and searches for multiple types of discrepancies after normalizing the results output by parsers. Distinct discrepancies are employed as feedback to update and prune the model. SBDT starts the next iteration from the updated and pruned model, unless all nodes and mutation operators have been pruned due to reaching their upper bounds. Our work has the following contributions: (1) To the best of our knowledge, this is the first time that testing of certificate parsers has been clearly distinguished from testing of certificate validators, which will facilitate accurate testing of certificate parsers and validators; (2) SBDT is the first systematic and efficient approach for differential testing of certificate parsers by searching, updating, and pruning models; and (3) We have implemented an open-source prototype tool of SBDT, and experimental results show that SBDT is effective and efficient in finding new bugs and enhancements of certificate parsers.

Interrupt-driven programs have been widely used in safety-critical areas such as aerospace and embedded systems. However, uncertain interleaving execution of interrupt service routines (ISRs) usually causes concurrency bugs. Specifically, when one or more ISRs attempt to preempt a sequence of instructions which are expected to be atomic, a kind of concurrency bugs namely atomicity violation may occur, and it is challenging to find this kind of bugs precisely and efficiently. In this paper, we propose a static approach for detecting atomicity violations in interrupt-driven programs. First, the program model is constructed with interruption points being selected to determine the possibly influenced ISRs. After that, reachability computation is conducted to build up a whole abstract reachability tree, and a delayed ISR-triggering strategy is employed to reduce the state space. Meanwhile, unserializable interleaving patterns are recognized to achieve the goal of atomicity violation detection. The approach has been implemented as a configurable tool namely CPA4AV. Extensive experiments show that CPA4AV is much more precise than the relative tools available with little extra time overhead. In addition, more complex situations can be dealt with CPA4AV.

As a lightweight method, runtime verification aims to check whether one program execution satisfies a desired property. For online runtime verification, the approach efficiency and property expressiveness are two key points restricting its wide application. In this paper, we propose a distributed network-based parallel runtime verification approach to verifying full regular temporal properties for a suitable subset of C (named by Xd-C) programs in an online manner. With this approach, an Xd-C program is translated into an equivalent Modeling, Simulation and Verification Language (MSVL) program, and a desired property is specified as a Propositional Projection Temporal Logic (PPTL) formula; during the program execution, segments of the generated state sequence are verified in parallel by distributed multi-core machines. Experimental results show that, our approach has a speedup of 2.5X-5.0X over the state-of-art runtime verification approaches and supports full regular temporal properties, meaning that our approach can not only take full advantage of computing and storage resources in a distributed network, but also support more expressive properties.

There is a wide agreement that model checking and automated planning (planning for short) are closely related fields. Planning is the task of finding a sequence of appropriate moves that achieves a goal. Model checking aims to prove or disprove a system model that satisfies a given property which is often specified by temporal logics. In this paper we investigate the application of advanced planning techniques to model checking. To this end, a system model is expressed by means of a planning model, and temporal logic property can be treated as a special form of planning goal, i.e., Temporally Extended Goal (TEG). Therefore, the model checking task can be reduced into a planning scheme what we call planning with TEG. In order to utilize the state-of-the-art planners, we further propose two novel compilation methods to translate a planning with TEG problem into a classical planning problem and a non-deterministic planning one respectively. The obtained valid plans in planning just correspond to the counterexamples in model checking. We provide detailed evaluations of our approach on a series of benchmarks. The experimental results are encouraging, showing that existing planners can provide significant improvements in the quality of the counterexamples compared with the model checkers.

The global trends in urbanization have caused many problems, among which Urban Traffic Control (UTC) becomes a priority issue for most big cities in many countries. As traffic demand changes rapidly, appropriate control policies are required to be generated in real time in order to, e.g., minimize congestion to reduce average travel time and air pollution. A practical way to meet the challenge is to build an intelligent control mechanism of road traffic. In this context, automated planning, a powerful and effective technique, can be exploited as an aid to dynamically produce plans to alleviate the problems of UTC. In this paper, we present an approach based on automated planning, in particular temporal planning scheme that aims for producing predictable management strategies of UTC. Meanwhile, a logic style control knowledge is employed to provide useful guidance for the search process in planning. We show the preliminary evaluations on simulation benchmarks closely related to UTC. Experimental results show the feasibility and effectiveness of our approach, compared with the state-of-the-art planners that participate in recent International Planning Competitions.

Knowledge based approaches developed for AI planning can convert an intractable planning problem to a tractable one. Current techniques often use temporal logics to express Search Control Knowledge (SCK) in logic based planning. However, traditional temporal logics are limited in expressiveness since they are unable to express spatial constraints which are as important as temporal ones in many planning domains. To this end, we propose a two-dimensional (spatial and temporal) logic namely PPTLSL by temporalizing separation logic with PPTL (Propositional Projection Temporal Logic) which is well-suited to specify SCK involving both spatial and temporal constraints in planning. We prove that PPTLSL is decidable essentially via an equisatisfiable translation from PPTLSL to its restricted form. Moreover, we implement a tool, S-TSolver, which effectively computes plans under the guidance of the spatio-temporal SCK expressed by PPTLSL formulas. The effectiveness of the tool is evaluated on selected benchmark domains from the International Planning Competition.

Temporal logics are widely adopted in Artificial Intelligence (AI) planning for specifying Search Control Knowledge (SCK). However, traditional temporal logics are limited in expressive power since they are unable to express spatial constraints which are as important as temporal ones in many planning domains. To this end, we propose a two-dimensional (spatial and temporal) logic namely PPTL^SL by temporalising separation logic with Propositional Projection Temporal Logic (PPTL). The new logic is well-suited for specifying SCK containing both spatial and temporal constraints which are useful in AI planning. We show that PPTL^SL is decidable and present a decision procedure. With this basis, a planner namely S-TSolver for computing plans based on the spatio-temporal SCK expressed in PPTL^SL formulas is developed. Evaluation on some selected benchmark domains shows the effectiveness of S-TSolver.