Neural architecture search (NAS) has achieved remarkable results in deep neural network design. Differentiable architecture search converts the search over discrete architectures into a hyperparameter optimization problem which can be solved by gradient descent. However, questions have been raised regarding the effectiveness and generalizability of gradient methods for solving non-convex architecture hyperparameter optimization problems. In this paper, we propose L2NAS, which learns to intelligently optimize and update architecture hyperparameters via an actor neural network based on the distribution of high-performing architectures in the search history. We introduce a quantile-driven training procedure which efficiently trains L2NAS in an actor-critic framework via continuous-action reinforcement learning. Experiments show that L2NAS achieves state-of-the-art results on NAS-Bench-201 benchmark as well as DARTS search space and Once-for-All MobileNetV3 search space. We also show that search policies generated by L2NAS are generalizable and transferable across different training datasets with minimal fine-tuning.

Neural architecture search automates neural network design and has achieved state-of-the-art results in many deep learning applications. While recent literature has focused on designing networks to maximize accuracy, little work has been conducted to understand the compatibility of architecture design spaces to varying hardware. In this paper, we analyze the neural blocks used to build Once-for-All (MobileNetV3), ProxylessNAS and ResNet families, in order to understand their predictive power and inference latency on various devices, including Huawei Kirin 9000 NPU, RTX 2080 Ti, AMD Threadripper 2990WX, and Samsung Note10. We introduce a methodology to quantify the friendliness of neural blocks to hardware and the impact of their placement in a macro network on overall network performance via only end-to-end measurements. Based on extensive profiling results, we derive design insights and apply them to hardware-specific search space reduction. We show that searching in the reduced search space generates better accuracy-latency Pareto frontiers than searching in the original search spaces, customizing architecture search according to the hardware. Moreover, insights derived from measurements lead to notably higher ImageNet top-1 scores on all search spaces investigated.

Despite the empirical success of neural architecture search (NAS) in deep learning applications, the optimality, reproducibility and cost of NAS schemes remain hard to assess. In this paper, we propose Generative Adversarial NAS (GA-NAS) with theoretically provable convergence guarantees, promoting stability and reproducibility in neural architecture search. Inspired by importance sampling, GA-NAS iteratively fits a generator to previously discovered top architectures, thus increasingly focusing on important parts of a large search space. Furthermore, we propose an efficient adversarial learning approach, where the generator is trained by reinforcement learning based on rewards provided by a discriminator, thus being able to explore the search space without evaluating a large number of architectures. Extensive experiments show that GA-NAS beats the best published results under several cases on three public NAS benchmarks. In the meantime, GA-NAS can handle ad-hoc search constraints and search spaces. We show that GA-NAS can be used to improve already optimized baselines found by other NAS methods, including EfficientNet and ProxylessNAS, in terms of ImageNet accuracy or the number of parameters, in their original search space.

This study investigates the problem of communication for a network composed of two half-duplex parallel relays with additive white Gaussian noise (AWGN). There is no direct link between the source and the destination. However, the relays can communicate with each other through the channel between them. Two protocols, i.e.,simultaneousandsuccessiverelaying, associated with two possible relay schedulings are proposed. The simultaneous relaying protocol is based on theBroadcast-Multiaccess with Common Message (BCM)scheme considered in. For the successive relaying protocol: (i) anon-cooperativescheme based on theDirty Paper Coding (DPC)and (ii) acooperativescheme based on theBlock Markov Encoding (BME)are considered. The composite scheme of employing BME inat mostone relay and DPC inat leastanother one is also proposed. It is proved that this scheme achieves at least the same rate when compared to thecooperativeandnon-cooperativeschemes for the Gaussian case. ASimultaneous-Successive Relaying based on Dirty Paper Coding scheme (SSRD)is also proposed. The optimum scheduling of the relays, and hence the capacity of the half-duplex Gaussian parallel relay channel in the low and high signal-to-noise ratio (SNR) scenarios, is derived. In the low SNR scenario, it is revealed that under certain conditions for the channel coefficients the ratio of the achievable rate of the simultaneous relaying based on BCM to the cut-set bound tends to be 1. On the other hand, as SNR goes to infinity it is proved that successive relaying, based on the DPC, asymptotically achieves the capacity of the network.

Schein and Gallager introduced the Gaussian parallel relay channel in 2000. They proposed the amplify-and-forward (AF) and the decode-and-forward (DF) strategies for this channel. For a long time, the best known achievable rate for this channel was based on the AF and DF with time sharing (AF-DF). A rematch-and-forward (RF) scheme for the scenario in which different amounts of bandwidth can be assigned to the first and second hops were proposed. In this paper, we propose acombinedamplify-and-decodeforward(CADF) scheme for the Gaussian parallel relay channel. We prove that the CADF scheme always gives a better achievable rate compared to the RF scheme, when there is a bandwidth mismatch between the first hop and the second hop. Furthermore, for the equal bandwidth case (Schein's setup), we show that the time sharing between the CADF and the DF schemes (CADF-DF) leads to a better achievable rate compared to the time sharing between the RF and the DF schemes (RF-DF) as well as the AF-DF.