The human brain has the ability to carry out new tasks with limited experience. It utilizes prior learning experiences to adapt the solution strategy to new domains. On the other hand, deep neural networks (DNNs) generally need large amounts of data and computational resources for training. However, this requirement is not met in many settings. To address these challenges, we propose the TUTOR (training neural networks using decision rules as model priors) DNN synthesis framework. TUTOR targets tabular datasets. It synthesizes accurate DNN models with limited available data and reduced memory/computational requirements. It consists of three sequential steps. The first step involves generation, verification, and labeling of synthetic data. The synthetic data generation module targets both categorical and continuous features. TUTOR generates synthetic data from the same probability distribution as the real data. It then verifies the integrity of the generated synthetic data using a semantic integrity classifier module. It labels synthetic data based on a set of rules extracted from the real dataset. Next, TUTOR uses two training schemes that combine synthetic and training data to learn the DNN model parameters. These two schemes focus on two different ways in which synthetic data can be used to derive a prior on the model parameters and, hence, provide a better DNN initialization for training with real data. In the third step, TUTOR employs a grow-and-prune synthesis paradigm to learn both the weights and the architecture of the DNN to reduce model size while ensuring its accuracy. We evaluate the performance of TUTOR on nine datasets of various sizes. We show that in comparison to fully connected DNNs, TUTOR, on an average, reduces the need for data by$5.9\times $(geometric mean), improves accuracy by 3.4%, and reduces the number of parameters (floating-point operations) by$4.7\times $($4.3\times $) (geometric mean). Thus, TUTOR enables less data-hungry, more accurate, and more compact DNN synthesis.

Many system design methods use population-based optimization or a surrogate model for solving constrained multiobjective optimization (CMOO). When designing a system with multiple objectives and constraints, the designer may first be interested in understanding the tradeoffs among different objectives from a small number of simulations. In the next step, the designer may focus on specific regions of interest in the design space near a set of nondominated solutions to further improve performance on the targeted objectives. This may help make the search process sample-efficient. We propose inverse design methodology for constrained multiobjective optimization (INFORM): a two-step approach for sample-efficient CMOO of real-world nonlinear systems. In the first step, we modify a genetic algorithm (GA) to make the design process sample-efficient. We inject candidate solutions into the GA population using inverse design methods instead of determining the candidate solutions for the next generation using only crossover and mutation, as is done in standard GA. We present three types of inverse design techniques based on: 1) a neural network (NN) verifier; 2) NN; and 3) Gaussian mixture model. The candidate solutions for the next generation are thus a mix of those generated using crossover/mutation and solutions generated using inverse design. At the end of the first step, we obtain a set of nondominated solutions. In the second step, we choose the regions of interest around the nondominated solutions to further improve the objective function values using inverse design methods. We demonstrate the efficacy of INFORM through synthesis of nonlinear systems and analog circuits. The experimental results show that INFORM reduces synthesis time by up to$29\times $and improves the value of the objective function by up to 33% compared to a state-of-the-art baseline design methodology.

Most system optimization techniques focus on finding the values of the system components to achieve the best performance. Searching over all component values gives the search methodology the freedom to explore the entire design space to determine the best system configuration. However, real-world systems often require searching in a restricted space over only a subset of component values while freezing some of the components to fixed values. Rather than optimizing from scratch to search over the subset of components, incorporating the past simulation logs (search performed when all components were allowed to vary) enables the optimization mechanism to utilize knowledge from past system behavior. In addition, when the system gives the same response over different combinations of input values, the designer may prefer one combination over another. Furthermore, real-world data often contain errors. To avoid catastrophic consequences of making decisions based on incorrect data points, we need a mechanism to identify and correct the resulting error. We propose REPAIRS, a methodology to complete/optimize partially specified systems. It also performs data integrity checks and identifies/corrects errors after detecting an anomaly in the data. We use a Gaussian mixture model to learn the joint distribution of the system inputs and the corresponding output response (objectives/constraints). We use the learned model to complete a partially specified system where only a subset of the component values and/or the system response is specified. When the system response exhibits multiple modes (e.g., same response for different combinations of input values), REPAIRS determines the combinations of input values that correspond to the several modes. Using past simulation logs, it searches over various subsets of system inputs to improve the performance of the reference solution. We also present a framework for verifying the integrity of a given data instance. When the integrity check fails, we provide a mechanism to identify the error location and correct the error. REPAIRS provides an explanation for the decision it makes for the different use cases described in this article. We provide results of REPAIRS in the context of completion, partial optimization, and data integrity check of real-world systems. REPAIRS achieves a hypervolume that is better than that obtained using a baseline method by up to 50%. It successfully identifies the error location and predicts the correct value of the erroneous feature with an error less than 0.2%. It detects error locations with a mean accuracy of up to 95% even when three feature values have an error.

Nonlinear system design is often a multiobjective optimization problem involving the search for a design that satisfies a number of predefined constraints. The design space is typically very large since it includes all possible system architectures with different combinations of components composing each architecture. In this article, we address nonlinear system design space exploration through a two-step approach encapsulated in a framework called fast design space exploration of nonlinear systems (ASSENT). In the first step, we use a genetic algorithm to search for system architectures that allow discrete choices for component values or else only component values for a fixed architecture. This step yields a coarse design since the system may or may not meet the target specifications. In contrast to prior works on design space exploration that rely on forward design, we use an inverse design in step 2 to search over a continuous space and fine-tune the component values with the goal of improving the value of the objective function. We use a neural network (NN) to model the system response. The NN is converted into a mixed-integer linear program for active learning to sample component values efficiently. We illustrate the efficacy of ASSENT on problems ranging from nonlinear system design to the design of electrical circuits. Experimental results show that ASSENT achieves the same or better value of the objective function compared to various other optimization techniques for nonlinear system design by up to 53%. We improve sample efficiency by 6–$12\times $compared to reinforcement learning-based synthesis of electrical circuits.