In reinforcement learning, the optimism in the face of uncertainty (OFU) is a mainstream principle for directing exploration towards less explored areas, characterized by higher uncertainty. However, in the presence of environmental stochasticity (noise), purely optimistic exploration may lead to excessive probing of high-noise areas, consequently impeding exploration efficiency. Hence, in exploring noisy environments, while optimism-driven exploration serves as a foundation, prudent attention to alleviating unnecessary over-exploration in high-noise areas becomes beneficial. In this work, we propose Optimistic Value Distribution Explorer (OVD-Explorer) to achieve a noise-aware optimistic exploration for continuous control. OVD-Explorer proposes a new measurement of the policy's exploration ability considering noise in optimistic perspectives, and leverages gradient ascent to drive exploration. Practically, OVD-Explorer can be easily integrated with continuous control RL algorithms. Extensive evaluations on the MuJoCo and GridChaos tasks demonstrate the superiority of OVD-Explorer in achieving noise-aware optimistic exploration.

Pre-routing timing prediction has been recently studied for evaluating the quality of a candidate cell placement in chip design. It involves directly estimating the timing metrics for both pin-level (slack, slew) and edge-level (net delay, cell delay), without time-consuming routing. However, it often suffers from signal decay and error accumulation due to the long timing paths in large-scale industrial circuits. To address these challenges, we propose a two-stage approach. First, we propose global circuit training to pre-train a graph auto-encoder that learns the global graph embedding from circuit netlist. Second, we use a novel node updating scheme for message passing on GCN, following the topological sorting sequence of the learned graph embedding and circuit graph. This scheme residually models the local time delay between two adjacent pins in the updating sequence, and extracts the lookup table information inside each cell via a new attention mechanism. To handle large-scale circuits efficiently, we introduce an order preserving partition scheme that reduces memory consumption while maintaining the topological dependencies. Experiments on 21 real world circuits achieve a new SOTA R2 of 0.93 for slack prediction, which is significantly surpasses 0.59 by previous SOTA method. Code will be available at: https://github.com/Thinklab-SJTU/EDA-AI.

Technology mapping is a crucial step in the logic synthesis in chip design e.g. Field Programmable Gate Arrays (FPGAs) design, where a logic network is transformed into a K-bounded lookup tables (K-LUTs) network. Traditional mapping algorithms converges quickly to a suboptimal result, which limits the exploration capacity for further improvement. In this paper, we propose a new mapping method called Exploration-enhanced heuristics and Adaptive sequencing for Technology Mapping (EasyMap). EasyMap includes a pool of new heuristics and considers the mapping exploration as a conditional sequence optimization problem. During the mapping exploration procedure, heuristic algorithms with specific parameters are selected and applied sequentially. Our EasyMap outperforms the widely used IfMap in ABC by a significant margin. In particular, when optimizing area with a level constraint, EasyMap outperforms IfMap by reducing 9.1% more area on arithmetic circuits of the EPFL benchmark. Moreover, when optimizing area without level constraints at the same time, EasyMap can reduce 19% more area than IfMap on arithmetic circuits.

Optimizing the quality of results (QoR) of a circuit during the logic synthesis (LS) phase in chip design is critical yet challenging. While most existing methods often mitigate the computational hardness by restricting the action space to a small set of operators and fixing the operator's parameters, they are susceptible to local minima and may not meet the high demand from industrial cases. In this paper, we develop a more comprehensive optimization approach via sample-efficient reinforcement learning (RL). Specifically, we first build a complete logic synthesis-RL environment, where the action space consists of three types of operators: logic optimization, technology mapping, and post-mapping, along with their associated continuouslbinary parameters for optimization as well. Based on this environment, we devise a hybrid proximal policy optimization (PPO) model to handle both discrete operators and parameters and design a distributed architecture to improve sample collection efficiency. Furthermore, we devise a dynamic exploration module to improve the exploration efficiency under the constraint of limited samples. We term our method as Exploration-enhanced RL for Logic Synthesis Sequence Optimization(EasySO). Results on the EPFL benchmark show that our method significantly outperforms current state-of-the-art models based on Bayesian optimization (BO) and the previous RL-based methods. Compared to resyn2, our EasySO achieves an average of 25.4% LUT-6 count optimization without sacrificing level values. Moreover, as of the time for this submission, we rank 26 first places among 40 optimization targets in the EPFL competition.

Logic synthesis (LS) is a process that transforms a high-level logic circuit description into a gate-level netlist, typically via a heuristic algorithm. Such a process can be decomposed into a series of transformation primitives, where each primitive optimizes the netlist while preserving the functional equivalence. However, identifying a desirable primitive sequence (PS) to achieve design goals is challenging, due to the immense design space. Recent advances in artificial intelligence offer the opportunity to leverage machine learning techniques to tackle the combinatorial optimization problem associated with PS. Unfortunately, the existing works either require time-consuming training for each circuit or incur high computational costs. To address these issues, we redefine the optimization of LS as a sequence generation problem and propose a generative pre-trained transformer (GPT) with offline reinforcement learning, which is named as GPT-LS. Thanks to the OpenABC-D dataset, GPT-LS is pre-trained on diverse circuits and its massive intermediate data during the synthesis, by utilizing the offline reinforcement learning technique of decision transformer. Then, GPT-LS is able to generate PS for unseen circuits to conduct optimized LS. According to our comprehensive experiments, GPT-LS achieves results that match those of previous state-of-the-art methods in a significantly shorter time. It is available at: github.com/Intelligent-Computing-Research-Group/GPT-LS.

Music recommendation is usually modeled as a Click-Through Rate (CTR) prediction problem, which estimates the probability of a user listening a recommended song. CTR prediction can be formulated as a binary classification problem where the played songs are labeled as positive samples and the skipped songs are labeled as negative samples. However, such naively defined labels are noisy and biased in practice, causing inaccurate model predictions. In this work, we first identify serious label noise issues in an industrial music App, and then propose an adaptive Label Correction method for Denoising (LCD) music recommendation by ensembling the noisy labels and the model outputs to encourage a consensus prediction. Extensive offline experiments are conducted to evaluate the effectiveness of LCD on both industrial and public datasets. Furthermore, in a one-week online AB test, LCD also significantly increases both the music play count and time per user by 1% to 5%.

During the logic synthesis flow of EDA, a sequence of graph transformation operators are applied to the circuits so that the Quality of Results (QoR) of the circuits highly depends on the chosen operators and their specific parameters in the sequence, making the search space operator-dependent and increasingly exponential. In this paper, we formulate the logic synthesis design space exploration as a conditional sequence optimization problem, where at each transformation step, an optimization operator is selected and its corresponding parameters are decided. To solve this problem, we propose a novel sequential black-box optimization approach without human intervention: 1) Due to the conditional and sequential structure of operator sequence with variable length, we build an embedding alignment cells based recurrent neural network as a surrogate model to estimate the QoR of the logic synthesis flow with historical data. 2) With the surrogate model, we construct acquisition function to balance exploration and exploitation with respect to each metric of the QoR. 3) We use multi-objective optimization algorithm to find the Pareto front of the acquisition functions, along which a batch of sequences, consisting of parameterized operators, are (randomly) selected to users for evaluation under the budget of computing resource. We repeat the above three steps until convergence or time limit. Experimental results on public EPFL benchmarks demonstrate the superiority of our approach over the expert-crafted optimization flows and other machine learning based methods. Compared to resyn2, we achieve 11.8% LUT-6 count descent improvements without sacrificing level values.

Gate Sizing is an important step in logic synthesis, where the cells are resized to optimize metrics such as area, timing, power, leakage, etc. In this work, we consider the gate sizing problem for leakage power optimization with timing constraints. Lagrangian Relaxation is a widely employed optimization method for gate sizing problems. We accelerate Lagrangian Relaxation-based algorithms by narrowing down the range of cells to resize. In particular, we formulate a heterogeneous directed graph to represent the timing graph, propose a heterogeneous graph neural network as the encoder, and train in the way of imitation learning to mimic the selection behavior of each iteration in Lagrangian Relaxation. This network is used to predict the set of cells that need to be changed during the optimization process of Lagrangian Relaxation. Experiments show that our accelerated gate sizer could achieve comparable performance to the baseline with an average of 22.5% runtime reduction.

Through using only a well-trained classifier, model-inversion (MI) attacks can recover the data used for training the classifier, leading to the privacy leakage of the training data. To defend against MI attacks, previous work utilizes a unilateral dependency optimization strategy, i.e., minimizing the dependency between inputs (i.e., features) and outputs (i.e., labels) during training the classifier. However, such a minimization process conflicts with minimizing the supervised loss that aims to maximize the dependency between inputs and outputs, causing an explicit trade-off between model robustness against MI attacks and model utility on classification tasks. In this paper, we aim to minimize the dependency between the latent representations and the inputs while maximizing the dependency between latent representations and the outputs, named a bilateral dependency optimization (BiDO) strategy. In particular, we use the dependency constraints as a universally applicable regularizer in addition to commonly used losses for deep neural networks (e.g., cross-entropy), which can be instantiated with appropriate dependency criteria according to different tasks. To verify the efficacy of our strategy, we propose two implementations of BiDO, by using two different dependency measures: BiDO with constrained covariance (BiDO-COCO) and BiDO with Hilbert-Schmidt Independence Criterion (BiDO-HSIC). Experiments show that BiDO achieves the state-of-the-art defense performance for a variety of datasets, classifiers, and MI attacks while suffering a minor classification-accuracy drop compared to the well-trained classifier with no defense, which lights up a novel road to defend against MI attacks.

Placement and routing are two critical yet time-consuming steps of chip design in modern VLSI systems. Distinct from traditional heuristic solvers, this paper on one hand proposes an RL-based model for mixed-size macro placement, which differs from existing learning-based placers that often consider the macro by coarse grid-based mask. While the standard cells are placed via gradient-based GPU acceleration. On the other hand, a one-shot conditional generative routing model, which is composed of a special-designed input-size-adapting generator and a bi-discriminator, is devised to perform one-shot routing to the pins within each net, and the order of nets to route is adaptively learned. Combining these techniques, we develop a flexible and efficient neural pipeline, which to our best knowledge, is the first joint placement and routing network without involving any traditional heuristic solver. Experimental results on chip design benchmarks showcase the effectiveness of our approach, with code that will be made publicly available.

Supervised learning under label noise has seen numerous advances recently, while existing theoretical findings and empirical results broadly build up on the class-conditional noise (CCN) assumption that the noise is independent of input features given the true label. In this work, we present a theoretical hypothesis testing and prove that noise in real-world dataset is unlikely to be CCN, which confirms that label noise should depend on the instance and justifies the urgent need to go beyond the CCN assumption.The theoretical results motivate us to study the more general and practical-relevant instance-dependent noise (IDN). To stimulate the development of theory and methodology on IDN, we formalize an algorithm to generate controllable IDN and present both theoretical and empirical evidence to show that IDN is semantically meaningful and challenging. As a primary attempt to combat IDN, we present a tiny algorithm termed self-evolution average label (SEAL), which not only stands out under IDN with various noise fractions, but also improves the generalization on real-world noise benchmark Clothing1M. Our code is released. Notably, our theoretical analysis in Section 2 provides rigorous motivations for studying IDN, which is an important topic that deserves more research attention in future.

For multi-class classification under class-conditional label noise, we prove that the accuracy metric itself can be robust. We concretize this finding's inspiration in two essential aspects: training and validation, with which we address critical issues in learning with noisy labels. For training, we show that maximizing training accuracy on sufficiently many noisy samples yields an approximately optimal classifier. For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping. Previously, model selection using noisy validation samples has not been theoretically justified. We verify our theoretical results and additional claims with extensive experiments. We show characterizations of models trained with noisy labels, motivated by our theoretical results, and verify the utility of a noisy validation set by showing the impressive performance of a framework termed noisy best teacher and student (NTS). Our code is released.

Given a fixed graph H, the H-free editing problem asks whether we can edit at most k edges to make a graph contain no induced copy of H. We obtain a polynomial kernel for this problem when H is a diamond. The incompressibility dichotomy for H being a 3-connected graph and the classical complexity dichotomy suggest that except for H being a complete/empty graph, H-free editing problems admit polynomial kernels only for a few small graphs H. Therefore, we believe that our result is an essential step toward a complete dichotomy on the compressibility of H-free editing. Additionally, we give a cubic-vertex kernel for the diamond-free edge deletion problem, which is far simpler than the previous kernel of the same size for the problem.

Block copolymer directed self-assembly (DSA) is a promising technique to print contacts/vias for the 10 nm technology node and beyond. By using hybrid lithography that incorporates DSA with multiple patterning, multiple masks are used to print the DSA templates and then the templates can be used to guide the self-assembly of the block copolymer. In this paper, we propose approaches to solve the simultaneous template optimization and mask assignment problem for DSA with multiple patterning. We verified in experiments that our approaches remarkably outperform the state-of-the-art work in reducing the manufacturing cost.

Block Copolymer Directed Self-Assembly (DSA) is a promising technique to print contacts/vias for the 10nm technology node and beyond. By using hybrid lithography that cooperates DSA with multiple patterning, multiple masks are used to print the DSA templates and then the templates can be used to guide the self-assembly of the block copolymer. In this paper, we propose approaches to solve the simultaneous template optimization and mask assignment problem for DSA with multiple patterning. We verified in experiments that our approaches remarkably outperform the state of the art work in reducing the manufacturing cost.