Design automation of complex analog circuits (CAC) with multiple sub-blocks is challenging mainly due to large design search space, uncertain intermediate subgoal creation, and lengthy CAC simulation runtime. In this work, we propose a hierarchical and heterogeneous integration framework as a fully automated and time-efficient CAC design optimization solution. In Particularly, we (i) decompose CAC into two levels hierarchically and for the first time introduce hierarchical RL agents with hindsight and subgoal testing to automate the subgoal creation between these two levels. The subgoal converges to the optimal value through algorithm interactions. (ii) We enable high-level design space dimensionality reduction, minimize CAC simulation runs through a buffer hold, and employ low-level sub-block execution parallelization to reduce overall runtime. (iii) We construct a heterogeneous integration of different RL algorithms and black-box optimization algorithms in hierarchy to further boost the speed by benefiting both from the hierarchical structure and the advantages of each different algorithm. Experiments on four CAC topologies demonstrate that this framework achieves a maximum of 11.4× speed up compared to existing methods at the desired figure-of-merit. This work opens up a time efficient design automation route for complex analog circuits and systems.

Despite the effort of analog circuit design automation, currently complex analog circuit design still requires extensive manual iterations, making it labor intensive and time-consuming. Recently, reinforcement learning (RL) algorithms have been demonstrated successfully for the analog circuit design optimization. However, a robust and highly efficient RL method to design analog circuits with complex design space has not been fully explored yet. In this work, inspired by multiagent planning theory as well as human expert design practice, we propose a multiagent based RL (MA-RL) framework to tackle this issue. Particularly, we (i) partition the complex analog circuits into several sub-blocks based on topology information and effectively reduce the complexity of design search space; (ii) leverage MA-RL for the circuit optimization, where each agent corresponds to a single sub-block, and the interactions between agents delicately mimic the best design tradeoffs between circuit sub-blocks by human experts; (iii) introduce and compare three different multiagent RL algorithms and corresponding frameworks to demonstrate the effectiveness of the MA-RL method. (iv) employing twin-delayed techniques and proximal policy to further boost training stability and accomplish higher performances. (v) The impacts of different reward function definitions as well as different state settings of MA-RL agents are investigated to further improve the robustness of this framework. (vi) Experiments on three different complex analog circuit topologies (GBA, DLL and SAR ADC) and knowledge transfers between two technology nodes are demonstrated. It’s shown that MA-RL framework can achieve the best FoM for complex analog circuits’ design. This work shines the light for future large scale analog circuit system design automation.