Adversarial robustness has emerged as an important topic in deep learning as carefully crafted attack samples can significantly disturb the performance of a model. Many recent methods have proposed to improve adversarial robustness by utilizing adversarial training or model distillation, which adds additional procedures to model training. In this paper, we propose a new training paradigm called Guided Complement Entropy (GCE) that is capable of achieving "adversarial defense for free," which involves no additional procedures in the process of improving adversarial robustness. In addition to maximizing model probabilities on the ground-truth class like cross-entropy, we neutralize its probabilities on the incorrect classes along with a "guided" term to balance between these two terms. We show in the experiments that our method achieves better model robustness with even better performance compared to the commonly used cross-entropy training objective. We also show that our method can be used orthogonal to adversarial training across well-known methods with noticeable robustness gain. To the best of our knowledge, our approach is the first one that improves model robustness without compromising performance.

Recent breakthroughs in Neural Architectural Search (NAS) have achieved state-of-the-art performance in many tasks such as image classification and language understanding. However, most existing works only optimize for model accuracy and largely ignore other important factors imposed by the underlying hardware and devices, such as latency and energy, when making inference. In this paper, we first introduce the problem of NAS and provide a survey on recent works. Then we deep dive into two recent advancements on extending NAS into multiple-objective frameworks: MONAS [19] and DPP-Net [10]. Both MONAS and DPP-Net are capable of optimizing accuracy and other objectives imposed by devices, searching for neural architectures that can be best deployed on a wide spectrum of devices: from embedded systems and mobile devices to workstations. Experimental results are poised to show that architectures found by MONAS and DPP-Net achieves Pareto optimality w.r.t the given objectives for various devices.

Power gating is one of the most effective ways to reduce leakage power. In this paper, we introduce a new relationship among maximum instantaneous current, IR-drops and sleep transistor networks from a temporal viewpoint. Based on this relationship, we propose an algorithm to reduce the total sizes of sleep transistors in distributed sleep transistor network designs with the consideration of decoupling capacitances is taken. Our method achieves significantly better results than previous works on sleep transistor sizes.

During the power mode transition, simultaneously turning on sleep transistors provides a sufficiently large surge current, which may cause a large IR drop in the power networks. The IR drop in turn causes errors in the retention sequential elements of the sleep modules or errors of the nonsleep modules. One efficient way to control the surge current is to schedule the turn-on sequences of sleep transistors. In this paper, we introduce several important properties of the surge current during the power mode transition for the distributed sleep transistor network (DSTN) design, which is a popular power gating design style. Based on these properties, we propose an accurate estimation of the surge current and provide efficient schedules on the DSTN structure. Our methods achieved significantly better results than previous works-on average, 261 times wake-up time reduction and 30% less energy loss during the power mode transition.

Power gating has been a very effective way to reduce leakage power. One important design issue for a power gating design is to limit the surge current during the wakeup process. Normally, a wakeup scheduling is required to control turn-on times of sleep transistors. In this paper, we adopt a voltage sensor to compare pre-designed reference voltages with the virtual ground voltage and use the comparison result to determine turn-on times of sleep transistors. Special properties and optimizations of using voltage sensors are discussed. Since a wakeup scheduling with fast wakeup time may require significant hardware resources, we propose a new wakeup scheduling formulation which considers the trade-off between wakeup times and hardware resources. Our experimental results show that with small increases on wakeup times, we can reduce significant hardware resources for a power gating design.

Power gating is one of the most effective ways to reduce leakage power. In this paper, we introduce a new relationship among Maximum Instantaneous Current, IR drops and sleep transistor networks from a temporal viewpoint. Based on this relationship, we propose an algorithm to reduce the total sizes of sleep transistors in Distributed Sleep Transistor Network designs. On average, the proposed method can achieve 21% reduction in the sleep transistor size.

During the power mode transition, a large surge current may lead to the malfunctions in a power-gating design. In this paper, we introduce several important properties of the surge current during the power mode transition for the distributed sleep transistor network (DSTN) designs. Based on these properties, we propose an accurate estimation of surge current and provide an efficient schedule on the DSTN structure. Our experiment achieved significantly better results than previous works - on average, 332 times wake-up time reduction and 35.48% less energy loss during the power mode transition.