We study the AutoML problem of automatically configuring machine learning pipelines by jointly selecting algorithms and their appropriate hyper-parameters for all steps in supervised learning pipelines. This black-box (gradient-free) optimization with mixed integer & continuous variables is a challenging problem. We propose a novel AutoML scheme by leveraging the alternating direction method of multipliers (ADMM). The proposed framework is able to (i) decompose the optimization problem into easier sub-problems that have a reduced number of variables and circumvent the challenge of mixed variable categories, and (ii) incorporate black-box constraints alongside the black-box optimization objective. We empirically evaluate the flexibility (in utilizing existing AutoML techniques), effectiveness (against open source AutoML toolkits), and unique capability (of executing AutoML with practically motivated black-box constraints) of our proposed scheme on a collection of binary classification data sets from UCI ML & OpenML repositories. We observe that on an average our framework provides significant gains in comparison to other AutoML frameworks (Auto-sklearn & TPOT), highlighting the practical advantages of this framework.

We consider the problem of removing and replacing clouds in satellite image sequences, which has a wide range of applications in remote sensing. Our approach first detects and removes the cloud-contaminated part of the image sequences. It then recovers the missing scenes from the clean parts using the proposed "TECROMAC" (TEmporally Contiguous RObust MAtrix Completion) objective. The objective function balances temporal smoothness with a low rank solution while staying close to the original observations. The matrix whose the rows are pixels and columns are days corresponding to the image, has low-rank because the pixels reflect land-types such as vegetation, roads and lakes and there are relatively few variations as a result. We provide efficient optimization algorithms for TECROMAC, so we can exploit images containing millions of pixels. Empirical results on real satellite image sequences, as well as simulated data, demonstrate that our approach is able to recover underlying images from heavily cloud-contaminated observations.

This paper surveys the state-of-the-art in continuous nonlinear optimization and makes the case that due to tremendous recent progress, larger and more complex problems can be solved than previously thought possible. The two basic paradigms, trust-region and line-search methods, are briefly described. In addition, various nonlinear optimization techniques are reviewed. The application of these nonlinear optimization methods to circuit sizing is presented by describing a pair of circuit sizing tools, one for dynamic tuning and one for tuning based on static timing analysis. Particular emphasis has been given to the customization of nonlinear optimization to the circuit sizing application.

Noise can cause digital circuits to switch incorrectly, producing spurious results. It can also have adverse power, timing and reliability effects. Dynamic logic is particularly susceptible to charge-sharing and coupling noise. Thus, the design and optimization of a circuit should take noise considerations into account. Such considerations are typically stated as semi-infinite constraints in the time-domain. Semi-infinite problems are generally harder to solve than standard nonlinear optimization problems. Moreover, the number of noise constraints can potentially be very large. This paper describes a novel and practical method for incorporating realistic noise considerations during automatic circuit optimization by representing semi-infinite constraints as ordinary equality constraints involving time integrals. Using an augmented Lagrangian optimization merit function, the adjoint method is applied to compute all the gradients required for optimization in a single adjoint analysis, no matter how many noise measurements are considered and irrespective of the dimensionality of the problem. Thus, for the first time, a method is described to practically accommodate a large number of noise considerations during circuit optimization. The technique has been applied to optimization using time-domain simulation, but could be applied in the future to optimization on a static-timing basis. Numerical results are presented.

Article Free Access Share on Gradient-based optimization of custom circuits using a static-timing formulation Authors: A. R. Conn IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NY IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NYView Profile , I. M. Elfadel IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NY IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NYView Profile , W. W. Molzen IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NY IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NYView Profile , P. R. O'Brien IBM Electronic Design Automation, 11400 Burnet Road, M. S. 9460, Austin, TX IBM Electronic Design Automation, 11400 Burnet Road, M. S. 9460, Austin, TXView Profile , P. N. Strenski IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NY IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NYView Profile , C. Visweswariah IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NY IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NYView Profile , C. B. Whan IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NY IBM Thomas J. Watson Research Center, Route 134 and Taconic, Yorktown Heights, NYView Profile Authors Info & Claims DAC '99: Proceedings of the 36th annual ACM/IEEE Design Automation ConferenceJune 1999 Pages 452–459https://doi.org/10.1145/309847.309979Published:01 June 1999Publication History 29citation367DownloadsMetricsTotal Citations29Total Downloads367Last 12 Months21Last 6 weeks3 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Static circuit optimization implies sizing of transistors and wires on a static timing basis, taking into account all paths through a circuit. Previous methods of formulating static circuit optimization produced problem statements that are very large and contain inherent redundancy and degeneracy. In this paper, a method of manipulating the timing formulation is presented which produces a dramatically more compact optimization problem, and reduces redundancy and degeneracy. The circuit optimization is therefore more efficient and effective. Numerical results to demonstrate these improvements are presented.

Noise can cause digital circuits to switch incorrectly and thus produce spurious results. Noise can also have adverse power, timing and reliability effects. Dynamic logic is particularly susceptible to charge-sharing and coupling noise. Thus the design and optimization of a circuit should take noise considerations into account. Such considerations are typically stated as semi-infinite constraints. In addition, the number of signals to be checked and the number of sub-intervals of time during which the checking must be performed can potentially be very large. Thus, the practical incorporation of noise constraints during circuit optimization is a hitherto unsolved problem. This paper

Automating the transistor and wire-sizing process is an important step toward being able to rapidly design high-performance, custom circuits. This paper presents a circuit optimization tool that automates the tuning task by means of state-of-the-art nonlinear optimization. It makes use of a fast circuit simulator and a general-purpose nonlinear optimization package. It includes minimax and power optimization, simultaneous transistor and wire tuning, general choices of objective functions and constraints, and recovery from nonworking circuits. In addition, the tool makes use of designer-friendly interfaces that automate the specification of the optimization task, the running of the optimizer, and the back-annotation of the results of optimization onto the circuit schematic. Particularly for large circuits, gradient computation is usually the bottleneck in the optimization procedure. In addition to traditional adjoint and direct methods, we use a technique called the adjoint Lagrangian method, which computes all the gradients necessary for one iteration of optimization in a single adjoint analysis. This paper describes the algorithms and the environment in which they are used and presents extensive circuit optimization results. A circuit with 6900 transistors, 4128 tunable transistors, and 60 independent parameters was optimized in about 108 min of CPU time on an IBM RISC/System 6000, model 590.

The circuit tuning problem is best approached by means of gradient-based nonlinear optimization algorithms. For large circuits, gradient computation can be the bottleneck in the optimization procedure. Traditionally, when the number of measurements is large relative to the number of tunable parameters, the direct method is used to repeatedly solve the associated sensitivity circuit to obtain all the necessary gradients. Likewise, when the parameters outnumber the measurements, the adjoint method is employed to solve the adjoint circuit repeatedly for each measurement to compute the sensitivities. In this paper we propose the adjoint Lagrangian method, which computes all the gradients necessary for augmented-Lagrangian-based optimization in a single adjoint analysis. After the nominal simulation of the circuit has been carried out, the gradients of the merit function are expressed as the gradients of a weighted sum of circuit measurements. The weights are dependent on the nominal solution and on optimizer quantities such as Lagrange multipliers. By suitably choosing the excitations of the adjoint circuit, the gradients of the merit function are computed via a single adjoint analysis, irrespective of the number of measurements and the number of parameters of the optimization. This procedure requires close integration between the nonlinear optimization software and the circuit simulation program.

Optimization of a circuit by transistor sizing is often a slow, tedious and iterative manual process which relies on designer intuition. Circuit simulation is carried out in the inner loop of this tuning procedure. Automating the transistor sizing process is an important step towards being able to rapidly design high-performance, custom circuits. JiffyTune is a new circuit optimization tool that automates the tuning task. Delay, rise/fall time, area and power targets are accommodated. Each (weighted) target can be either a constraint or an objective function. Minimax optimization is supported. Transistors can be ratioed and similar structures grouped to ensure regular layouts. Bounds on transistor widths are supported. JiffyTune uses LANCELOT, a large-scale nonlinear optimization package with an augmented Lagrangian formulation. Simple bounds are handled explicitly and trust region methods are applied to minimize a composite objective function. In the inner loop of the optimization, the fast circuit simulator SPECS is used to evaluate the circuit. SPECS is unique in its ability to efficiently provide time-domain sensitivities, thereby enabling gradient-based optimization. Both the adjoint and direct methods of sensitivity computation have been implemented in SPECS. To assist the user, interfaces in the Cadence and SLED design systems have been constructed.

Consider a system of k components that fails whenever there is a defect in at least one of the components. Due to cost and time constraints it is not feasible to learn exactly which components are defective. Instead, test procedures ascertain that the defective components belong to some subset of the k components. This phenomenon is termed masking. The authors describe a 2-stage procedure in which a sample of masked subsets is subjected to intensive failure analysis. This enables maximum-likelihood estimation of the defect probability of each individual component and leads to diagnosis of the defective components in future masked failures.

The purpose of this article is to discuss the scope and functionality of a versatile environment for testing small- and large-scale nonlinear optimization algorithms. Although many of these facilities were originally produced by the authors in conjunction with the software package LANCELOT, we believe that they will be useful in their own right and should be available to researchers for their development of optimization software. The tools can be obtained by anonymous ftp from a number of sources and may, in many cases, be installed automatically. The scope of a major collection of test problems written in the standard input format (SIF) used by the LANCELOT software package is described. Recognizing that most software was not written with the SIF in mind, we provide tools to assist in building an interface between this input format and other optimization packages. These tools provide a link between the SIF and a number of existing packages, including MINOS and OSL. Additionally, as each problem includes a specific classification that is designed to be useful in identifying particular classes of problems, facilities are provided to build and manage a database of this information. There is a Unix and C shell bias to many of the descriptions in the article, since, for the sake of simplicity, we do not illustrate everything in its fullest generality. We trust that the majority of potential users are sufficiently familiar with Unix that these examples will not lead to undue confusion.

An algorithm for solving the problem minimize II A T x --bill subject to optional constraints of the form GTx ffi h, CTx >_ d, is given, together with an waplementation m Fortran.The algorithm generalizes one given earlier by the same authors for the unconstrained l~ problem.

indicates a Fortran module with n records NAMET(n): indicates "NAME" is included for testing purposes NAMED(n): indicates "NAME" contains test data