Large Language Models (LLMs) are increasingly applied to data-centric tasks in software maintenance and evolution, such as quality assurance and migration. While recent methods constrain LLM outputs using grammars or regular expressions, these syntactic techniques fail to enforce deeper semantic constraints involving numeric dependencies, conditional logic, and checksums. We present ClauseBandit, a framework that combines LLMs with Satisfiability Modulo Theories (SMT) solvers to generate structured data satisfying such constraints from natural language specifications. ClauseBandit introduces a Bayesian inference approach that selects the most plausible SMT formula using posterior probabilities derived from formula self-consistency and data likelihoods. Evaluated on 27 structured generation tasks inspired by industrial use-cases, ClauseBandit successfully selected valid SMT formulas for$\mathbf{7 4. 1 \%}$of tasks. Our approach enables LLM-based structured generation that goes beyond syntax, producing semantically valid, reusable constraint specifications from natural language.

In out sourced industrial projects for testing of web applications, often neither the application to be tested, nor its source code are provided to the testing team, due to confidentiality reasons, making systematic testing of these applications very challenging. However, textual descriptions of such systems are often available. So, one can consider leveraging a Large Language Model (LLM) to parse these descriptions and synthesize test generators (programs that produce test data). In our experience, LLM synthesized test generators suffer from two problems:- (1) unsound: the generators might produce invalid data and (2) incomplete: the generators typically fail to generate all expected valid inputs. To mitigate these problems, we introduce TestRefineGen a method for autonomously generating test data from textual descriptions. TestRe-fineGen begins by invoking an LLM to parse a given corpus of documents and produce multiple test gener-ators. It then uses a novel ranking approach to identify generators that can produce invalid test data, and then automatically repairs them using a counterexample-guided refinement process. Lastly, TestRefineGen per-forms a generalization procedure that offsets synthesis or refinements that leads to incompleteness, to obtain generators that produce more comprehensive valid in-puts. We evaluated the effectiveness of TestRefineGen on a manually curated set of 256 textual descriptions of test data. TestRefineGen synthesized generators that produce valid test data for 66.01 % of the descriptions. Using a combination of post-processing sanitisation and refinement it was able to successfully repair synthesized generators, which improved the success rate to 76.95 %. Further, our statistical analysis on a small subset of synthesized generators shows that TestRefineGen is able to generate test data that is well distributed across the input space. Thus, TestRefineGen can be an effective technique for autonomous test data generation for web testing in projects with confidentiality concerns.

Heating, ventilation, and air-conditioning (HVAC) system’s supervisory control is crucial for energy-efficient thermal comfort in buildings. The control logic is usually specified as ‘if-then-that-else’ rules that capture the domain expertise of HVAC operators, but they often have conflicts that may lead to sub-optimal HVAC performance. We propose EImprove, a reinforcement-learning (RL) based framework that exploits these conflicts to learn a resolution policy. We evaluate EImprove through a co-simulation strategy involving EnergyPlus simulations of a real-world office setting and a formal requirement specifier. Our experiments show that EImprove learns 75% faster than a pure RL framework.

Conventional automated test case generation techniques do not scale to modern software systems, as these systems have a large number of requirements that change frequently. In this paper, we present a scalable algorithm, AGenT, that generates test cases to cover maximal requirements. AGenT takes Expressive Decision Tables (EDT), specifying requirements of a system, as input and realises these as multiple Discrete Time Automata (DTAs). AGenT then generates test cases to cover each row of the tables. To improve scalability, it attempts to cover nearer rows (requiring fewer inputs) first, where distance is measured using a novel distance-to-match heuristic. It also maintains information about desirability and predictability of inputs so as to select promising inputs with a higher probability. Although the algorithm has been presented in the context of EDT, it operates on its DTA representation and hence can be applied to any system that is represented as a collection of DTAs like Statemate and Stateflow. In this paper, we describe AGenT in detail and present findings from two experiments that we conducted. We compared AGenT with state-of-the-art algorithms, DRAFT and a random test case generation algorithm, RTG. In the first experiment, AGenT took a maximum of 144 seconds to cover all rows whereas the other two algorithms timed out on many modules. In the second experiment, for a module with 701 rows, AGenT achieved 7% more coverage than DRAFT and 12% more than RTG.

Creating test cases to cover all functional requirements of real-world systems is hard, even for domain experts. Any method to generate functional test cases must have three attributes: (a) an easy-to-use formal notation to specify requirements, from a practitioner's point of view, (b) a scalable test-generation algorithm, and (c) coverage criteria that map to requirements. In this paper we present a method that has all these attributes. First, it includes Expressive Decision Table (EDT), a requirement specification notation designed to reduce translation efforts. Second, it implements a novel scalable row-guided random algorithm with fuzzing (RGRaF)(pronounced R-graph) to generate test cases. Finally, it implements two new coverage criteria targeted at requirements and requirement interactions. To evaluate our method, we conducted experiments on three real-world applications. In these experiments, RGRaF achieved better coverage than pure random test case generation. When compared with manual approach, our test cases subsumed all manual test cases and achieved up to 60% effort savings. More importantly, our test cases, when run on code, uncovered a bug in a post-production sub-system and captured three missing requirements in another.

Requirements of reactive systems express the relationship between sensors and actuators and are usually described in a natural language and a mix of state-based and stream-based paradigms. Translating these into a formal language is an important pre-requisite to automate the verification of requirements. The analysis effort required for the translation is a prime hurdle to formalization gaining acceptance among software engineers and testers. We present Expressive Decision Tables (EDT), a novel formal notation designed to reduce the translation efforts from both state-based and stream-based informal requirements. We have also built a tool, EDTTool, to generate test data and expected output from EDT specifications. In a case study consisting of more than 200 informal requirements of a real-life automotive application, translation of the informal requirements into EDT needed 43% lesser time than their translation into Statecharts. Further, we tested the Statecharts using test data generated by EDTTool from the corresponding EDT specifications. This testing detected one bug in a mature feature and exposed several missing requirements in another. The paper presents the EDT notation, comparison to other similar notations and the details of the case study.