Image restorations for single degradations have been widely studied, demonstrating excellent performance for each degradation, but can not reflect unpredictable realistic environments with unknown multiple degradations, which may change over time. To mitigate this issue, image restorations for known and unknown multiple degradations have recently been investigated, showing promising results, but require large networks or have sub-optimal architectures for potential interference among different degradations. Here, inspired by the filter attribution integrated gradients (FAIG), we propose an adaptive discriminative filter-based model for specific degradations (ADMS) to restore images with unknown degradations. Our method allows the network to contain degradation-dedicated filters only for about 3% of all network parameters per each degradation and to apply them adaptively via degradation classification (DC) to explicitly disentangle the network for multiple degradations. Our proposed method has demonstrated its effectiveness in comparison studies and achieved state-of-the-art performance in all-in-one image restoration benchmark datasets of both Rain-Noise-Blur and Rain-Snow-Haze.

As the physical size of recent CMOS image sensors (CIS) gets smaller, the latest mobile cameras adopt unique non-Bayer color filter array (CFA) patterns (e.g., Quad, Nona, Q×Q), which consist of homogeneous color units with adjacent pixels. These non-Bayer CFAs are superior to conventional Bayer CFA thanks to their changeable pixel-bin sizes for different light conditions, but may introduce visual artifacts during demosaicing due to their inherent pixel pattern structures and sensor hardware characteristics. Previous demosaicing methods have primarily focused on Bayer CFA, necessitating distinct reconstruction methods for non-Bayer CIS with various CFA modes under different lighting conditions. In this work, we propose an efficient unified demosaicing method that can be applied to both conventional Bayer RAW and various non-Bayer CFAs’ RAW data in different operation modes. Our Knowledge Learning-based demosaicing model for Adaptive Patterns, namely KLAP, utilizes CFA-adaptive filters for only 1% key filters in the network for each CFA, but still manages to effectively de-mosaic all the CFAs, yielding comparable performance to the large-scale models. Furthermore, by employing meta-learning during inference (KLAP-M), our model is able to eliminate unknown sensor-generic artifacts in real RAW data, effectively bridging the gap between synthetic images and real sensor RAW. Our KLAP and KLAP-M methods achieved state-of-the-art demosaicing performance in both synthetic and real RAW data of Bayer and non-Bayer CFAs.

In advanced nodes, scaling of critical dimension and pitch has not progressed at historical Moore’s Law rates. Thus,scaling boostersare explored to improve achievable power, performance, area, and cost (PPAC) in new technologies. However, scaling boosters increase complexity of standard-cell architectures, power delivery, design rules, and other aspects of the design enablement, and may not result in design-level benefits. Therefore, design-technology co-optimization (DTCO) methodologies are required to evaluate design-level benefits of scaling boosters. The key challenge for DTCO is that large engineering efforts and long timelines are needed to develop design enablements (e.g., cell libraries) and perform implementation studies in order to assess technology options. We describe a new framework that can systematically evaluate a measure of intrinsic routability,$K_{\mathrm{ th}}$, across both technology and design choices. We focus on routability since it is a critical factor in the scaling of area and cost. Our framework includes realistic standard-cell libraries that are automatically generated using satisfiability modulo theory (SMT) methods, and a new pin shape selection method. Routability assessments are based on the PROBE approach and an improved construction of underlying netlist topologies. Our experimental studies demonstrate the assessment of routability impacts for advanced-node technology and design options. We demonstrate learning-based$K_{\mathrm{ th}}$prediction to reduce runtime, disk space and commercial tool licenses needed to implement our framework. Our work enables faster and more comprehensive evaluation of technology options early in the technology development process.

Often, labeling large amount of data is challenging due to high labeling cost limiting the application domain of deep learning techniques. Active learning (AL) tackles this by querying the most informative samples to be annotated among unlabeled pool. Two promising directions for AL that have been recently explored are task-agnostic approach to select data points that are far from the current labeled pool and task-aware approach that relies on the perspective of task model. Unfortunately, the former does not exploit structures from tasks and the latter does not seem to well-utilize overall data distribution. Here, we propose task-aware variational adversarial AL (TA-VAAL) that modifies task-agnostic VAAL, that considered data distribution of both label and unlabeled pools, by relaxing task learning loss prediction to ranking loss prediction and by using ranking conditional generative adversarial network to embed normalized ranking loss information on VAAL. Our proposed TA-VAAL outperforms state-of-the-arts on various benchmark datasets for classifications with balanced / imbalanced labels as well as semantic segmentation and its task-aware and task-agnostic AL properties were confirmed with our in-depth analyses.

With the relentless scaling of technology nodes, physical design engineers encounter non-trivial challenges caused by rapidly increasing design complexity, particularly in the routing stage. Back-end designers must manually stitch/modify all of the design rule violations (DRVs) that remain after automatic place-and-route (P&R), during the implementation of engineering change orders (ECOs). In this paper, we propose CoRe-ECO, a concurrent refinement framework for efficient automation of the ECO process. Our framework efficiently resolves pin accessibility-induced DRVs by simultaneously performing detailed placement, detailed routing, and cell replacement. In addition to perturbation-minimized solutions, our proposed SMT-based optimization framework also suggests the adoption of alternative master cells to better achieve DRV-clean layouts. We demonstrate that our framework successfully resolves from 33.3% to 100.0% (58.6% on average) of remaining DRVs on M1-M3 layers, across a range of benchmark circuits with various cell architectures, while also providing average total wirelength reduction of 0.003%.

In this article, we propose an automated standard cell synthesis framework, SP&R, which simultaneously solves P&R without deploying any sequential/separate operations, by a novel dynamic pin allocation scheme. The proposed SP&R utilizes the multiobjective optimization feature of satisfiability modulo theories (SMT) to obtain optimal cell layouts. To achieve practical scalability of the framework, we develop various search-space reduction techniques, including breaking symmetry, conditional assignment/localization, and cell/objective function partitioning. Compared to the previous work, SP&R achieves 20.8× to 131.7× runtime improvements on average across the design-rule sets. As a result, SP&R successfully produces cell layouts up to 36 field-effect transistors (FETs) and 27 nets within 1.75 h by orchestrating all innovative tactics together, resulting in the generation of a whole 7-nm standard cell library. Compared to the known layouts, our work improves cell size and # M2 tracks by 0.1 contacted poly pitch and 0.3 tracks, respectively.

With the relentless scaling of technology nodes, design technology co-optimization (DTCO) for the conventional (Conv.) cell structure is starting to reach its limitations due to limited routing resources, lateral p-n separations, and performance requirements. As a result, system technology co-optimization (STCO) has been proposed to exploit the benefits of 3-D architectures. Complementary-FET (CFET) technology, which stacks p-FET on n-FET or vice versa, can release the restriction of p-n separation and reduce in-cell routing congestion by enabling p-n direct connections. However, CFET standard cell (SDC) synthesis demands holistic considerations to maximize the area benefit of scaling at the block level due to the extremely limited routability that comes from the stacked structure and reduced cell height. In this article, we propose a satisfiability modulo theory (SMT)-based CFET SDC synthesis framework that simultaneously solves place-and-route to generate optimized layouts. We first demonstrate that the CFET structure achieves 10.94% and 21.27% reduction on average cell area and metal length, respectively, and 15.10% smaller block-level area compared to Conv. structure as scaling down to 3.5T architecture. For routability, the proposed constraint-based minimum pin length/minimum pin opening and objective-based edge-based pin-separation/M2 track use reduce up to 48% #DRVs at the block level compared to the previous work. Then, through extensive DTCO explorations on ground design rules and #BEOLs, 3.5T CFET SDCs achieve up to 6.50% smaller block-level areas than 4.5T CFET SDCs. Finally, with the assistance of STCO and DTCO, 3.5T CFET SDCs achieve 21.0% on average reduced block-level areas compared to 4.5T Conv. SDCs.

Standard cell synthesis requires careful engineering approaches to ensure routability across various digital IC designs since physical design (PD) for sub-7nm technology nodes demands holistic efforts to address urgent and nontrivial design challenges. The smaller number of routing tracks and more complex design rules due to the sophisticated multi-patterning technology make place-and-route (P&R) for designing a standard cell extremely hard and time-consuming. Many conventional approaches have been suggested for improving transistor-level P&R and pin accessibility, nonetheless insufficient because of the heuristic/divide-and-conquer manners. In this paper, we propose a novel framework, SP&R, which simultaneously solves P&R for designing standard cell's layout without deploying any sequential procedures (between place and route steps) by using dynamic pin allocation-based cell synthesis. The proposed SP&R utilizes the Optimization Modulo Theories (OMT), an extension of the Satisfiability modulo theories (SMT), to obtain optimal standard cell layout by virtue of SAT (Boolean Satisfiability)-based fast reasoning ability. We validate that our SP&R framework achieves 10.5% of reduction on average in terms of metal length compared to the sequential approach, through practical standard cell designs targeting sub-7nm technology nodes.

As the technology node is evolving, standard cell (SDC) design scaling is obstructed by design constraints such as limited routing resources, lateral P-N separation, and performance requirements. Complimentary-FET (CFET) technology, which stacks the P-FET on N-FET or vice versa, is able to release the restriction of P-N connection for SDC layout scaling. However, (both in-cell and block-level) routable CFET SDC design, while maintaining the scaling advantages, is a non-trivial problem because of the extremely limited routability (including pin-accessibility) comes from the intrinsic stacked FET structure.

Rotation invariance has been an important topic in computer vision tasks. Ideally, robot grasp detection should be rotation-invariant. However, rotation-invariance in robotic grasp detection has been only recently studied by using rotation anchor box that are often time-consuming and unreliable for multiple objects. In this paper, we propose a rotation ensemble module (REM) for robotic grasp detection using convolutions that rotates network weights. Our proposed REM was able to outperform current state-of-the-art methods by achieving up to 99.2% (image-wise), 98.6% (object-wise) accuracies on the Cornell dataset with real-time computation (50 frames per second). Our proposed method was also able to yield reliable grasps for multiple objects and up to 93.8% success rate for the real-time robotic grasping task with a 4-axis robot arm for small novel objects that was significantly higher than the baseline methods by 11-56%.

Applications of deep neural network (DNN) based object and grasp detections could be expanded significantly when the network output is processed by a high-level reasoning over relationship of objects. Recently, robotic grasp detection and object detection with reasoning have been investigated using DNNs. There have been efforts to combine these multitasks using separate networks so that robots can deal with situations of grasping specific target objects in the cluttered, stacked, complex piles of novel objects from a single RGB-D camera. We propose a single multi-task DNN that yields accurate detections of objects, grasp position and relationship reasoning among objects. Our proposed methods yield state-of-the-art performance with the accuracy of 98.6% and 74.2% with the computation speed of 33 and 62 frame per second on VMRD and Cornell datasets, respectively. Our methods also yielded 95.3% grasp success rate for novel object grasping tasks with a 4-axis robot arm and 86.7% grasp success rate in cluttered novel objects with a humanoid robot.

With the scaling of VLSI technologies, the design-technology co-optimization (DTCO) requires prompt development of standard cell libraries to explore scaling effects of various cell architectures. However, standard cell layout design demands holistic efforts for processing transistor placement and in-cell routing due to the limited routing tracks and complicated design rules. Thus, an automatic design framework of standard cell layout became essential in the advanced scaling. Conventional heuristic/divide-and-conquer approaches lack the optimality of solutions because of the limited solution space. In this paper, we propose a novel standard cell scaling framework that simultaneously finds an optimal solution in placement and routing with the pin-accessibility. To ensure the minimum number of pin-access points, we devise strict Boolean counter-based design constraints. We validate our framework using scaling parameters and cell architectures across sub-7nm technology nodes.

Pin accessibility encounters nontrivial challenges due to the smaller number of routing tracks, higher pin density, and more complex design rules. Consequently, securing design rule-correct routability has become a critical bottleneck for sub-10-nm IC designs (particularly in the detailed routing stage) costing days of runtime. To reduce turnaround time, IC designers demand new design methodologies to analyze the routing feasibility of a given layout architecture (e.g., conditional design rules, pin assignment patterns, etc). There are several conventional methods capable of assessing routability that consider pin accessibility. However, precise diagnosis of unroutable layouts remains an open problem for IC design practitioners. In this article, we propose two novel frameworks that: 1) efficiently analyzes design rule-correct routability via an integer linear programming (ILP)-derived Boolean satisfiability (SAT) formulation written in light-weight conjunctive normal form, on top of multicommodity flow theory and 2) precisely diagnose explicit reasons for design-rule violations (DRVs) in the form of human-interpretable explanations, while specifying conflicting design rules with a physical location. While covering a variety of conditional design rules, we have refined our formulation by using SAT encoding techniques, supernode simplification, Boolean constraint propagation-based preprocessing, etc. We demonstrate that our routability analysis framework produces design rule-correct routability assessment within 0.02% of ILP runtime on average. Also, our routability diagnosis framework precisely examines DRVs, revealing design-rule conflicts for a variety of pin layouts and switchboxes. We show our frameworks scalability by utilizing practical benchmarks ranging up to 40000 grid-size layouts (i.e., 200 Htrack × 200 Vtrack), producing results within an hour.

Routability diagnosis has increasingly become the bottleneck in detailed routing for sub-10nm technology due to the limited tracks, high density, and complex design rules. The conventional ways to examine the routability of detailed routing are ILP- and SAT-based techniques. However, once we identify the routability, the diagnosis remains an open problem for physical designers. In this paper, we propose a novel framework, called ROAD, which diagnoses explicit reasons for routing failures. The proposed ROAD framework utilizes a diagnosis-friendly SAT formulation to represent design's layout and diagnoses the routability with SAT solving techniques. Based on the diagnosis, ROAD provides human-interpretable explanations for conflicted routing conditions. To show the practical value of our framework, we also generate comprehensive test-sets that enable exhaustive exploration of layouts based on Rent's rule. We demonstrate that ROAD successfully examines conflict causes for diverse pin layouts. Throughout extensive diagnosis, we also present several key findings for design failure. ROAD performs routability diagnosis within 2 minutes on average for 90 grids testsets, while diagnosing the exact causes of routing failures in terms of congestion and conditional design rules.

Three-dimensional integrated circuit (3D IC) technology offers a potential breakthrough to enable a paradigm-shift strategy, called “more than Moore,” with novel features and advantages over the conventional 2D process technology. By having three-dimensional interconnections, 3D IC provides substantial wirelength reduction and a massive amount of bandwidth, which gives significant performance improvement to overcome many of the nontrivial challenges in semiconductor industry. Moreover, 3D integration technology enables to stack disparate technologies with various functionalities into a single system-in-package (SiP), introducing “true 3D IC” design. As the first physical design (PD) step, IC floorplanning takes a crucial role to determine IC’s overall design qualities such as footprint area, timing closure, power distribution, thermal management, and so on. However, lack of efficient 3D floorplanning algorithms that practically implement advantages of 3D integration technology is a critical bottleneck for PD automation of 3D IC design and implementation. 3D floorplanning (or packing, block partitioning) is a well-known NP-hard problem, and most of 3D floorplanning algorithms rely on heuristics and iterative improvements. Thus, developing complete and efficient 3D floorplan representations is important, since floorplan representation provides the foundation of data structure to search the solution space for 3D IC floorplanning. A well-defined floorplan representation provides a well-organized and cost-effective methodology to design high-performance 3D IC. We propose a new 3D IC floorplan representation methodology using corner links and partial order . Given a fixed number of cuboidal blocks and their volume, algorithmic 3D floorplan representations describe topological structure and physical positions/orientations of each block relative to the origin in the 3D floorplan space. In this article, (1) we introduce our novel 3D floorplan representation, called corner links representation , (2) we analyze the equivalence relation between the corner links representation and its corresponding partial order representation , and (3) we discuss several key properties of the corner links representation and partial order representation. The corner links representation provides a complete and efficient structure to assemble the original 3D mosaic floorplan. Also, the corner links representation for the non-degenerate 3D mosaic floorplan can be equivalently expressed by the four trees representation . The partial order representation defines the topological structure of the 3D floorplan with three transitive closure graphs (TCG) for each direction and captures all stitching planes in the 3D floorplan in the order of their respective directions. We demonstrate that the corner links representation can be reduced to its corresponding partial order representation, indicating that the corner links representation shares well-defined and -studied features/properties of 3D TCG-based floorplan representation. If the partial order representation describes relations between any pairs of blocks in the 3D floorplan, then the floorplan is a valid floorplan. We show that the partial order representation can restore the absolute coordinates of all blocks in the 3D mosaic floorplan by using the given physical dimensions of blocks.

Transient simulation becomes a bottleneck for modern IC designs due to large numbers of transistors, interconnects and tight design margins. For modified nodal analysis (MNA) formulation, we could have differential algebraic equations (DAEs) which consist ordinary differential equations (ODEs) and algebraic equations. Study of solving DAEs with conventional multi-step integration methods has been a research topic in the last few decades. We adopt matrix exponential based integration method for circuit transient analysis, its stability and accuracy with DAEs remain an open problem. We identify that potential stability issues in the calculation of matrix exponential and vector product (MEVP) with rational Krylov method are originated from the singular system matrix in DAEs. We then devise a robust algorithm to implicitly regularize the system matrix while maintaining its sparsity. With the new approach, $\varphi$ functions are applied for MEVP to improve the accuracy of results. Moreover our framework no longer suffers from the limitation on step sizes thus a large leap step is adopted to skip many simulation steps in between. Features of the algorithm are validated on large-scale power delivery networks which achieve high efficiency and accuracy.

Tree structures and algorithms provide a fundamental and powerful data abstraction and methods for computer science and operations research. In particular, they enable significant advancement of IC physical design techniques and design optimization. For the last half century, Prof. T. C. Hu has areas in computer science, including network flows, integer programming, shortest paths, binary trees, global routing, etc. In this article, we select and summarize three important and interesting tree-related topics (ancestor trees, column generation, and alphabetical trees) in the highlights of Prof. T. C. Hu's contributions to physical design.

In this paper, we propose a novel synthesis algorithm for time-varying transformations of signals With dispersive structures or that have been transmitted in highly non-linear dispersive mediums. Our algorithms are based on transformation techniques that counteract the effects of non-linear dispersive group delay changes by converting them to constant or linear group delay changes. In particular, we apply the algorithm to synthesize a signal with hyperbolic group delay structure from its Altes Q-distribution (QD). Such algorithms are important in recovering relevant dispersive signal segments after processing such as filtering in the time-frequency plane. We demonstrate the importance of our new synthesis algorithms in suppressing dispersive time-varying interference or impulsive artifacts in dispersive mediums, such as the ocean, in wireless communications applications. We demonstrate that the new approach using synthesis of dispersive structures successfully improved the system's bit error rate performance when compared to the use of techniques that use synthesis of linear structures.