Choices of dataflows, which are known as intra-core neural network (NN) computation loop nest scheduling and inter-core hardware mapping strategies, play a critical role in the performance and energy efficiency of NoC-based neural network accelerators. Confronted with an enormous dataflow exploration space, this paper proposes an automatic framework for generating and optimizing the full-layer-mappings based on two reinforcement learning algorithms including A2C and PPO. Combining soft and hard constraints, this work transforms the mapping configuration into a sequential decision problem and aims to explore the performance and energy efficient hardware mapping for NoC systems. We evaluate the performance of the proposed framework on 10 experimental neural networks. The results show that compared with the direct-X mapping, the direct-Y mapping, GA-base mapping, and NN-aware mapping, our optimization framework reduces the average execution time of 10 experimental NNs by 9.09$\%$, improves the throughput by 11.27$\%$, reduces the energy by 12.62$\%$, and reduces the time-energy-product (TEP) by 14.49$\%$. The results also show that the performance enhancement is related to the coefficient of variation of the neural network to be computed.

Network-on-Chip (NoC) is a scalable on-chip communication architecture for the NN accelerator, but with the increase in the number of nodes, the communication delay becomes higher. Applications such as machine learning have a certain resilience to noisy/erroneous transmitted data. Therefore, approximate communication becomes a promising solution to improving performance by reducing traffic loads under the constraint of the acceptable maximum accuracy loss of neural networks. It is a key issue to balance the result quality and the communication delay for approximate NoC systems. The traditional approximate NoC only considers the node-to-node approximation-based dynamic traffic regulation. However, the dynamically changing traffic patterns across different nodes, different times, and different applications lead to a huge search space, which makes it hard to explore an optimal global approximation solution. In this paper, we propose a quality model for different neural networks, which presents the relationship between the quality loss and the data approximate rate. Then, a hierarchical approximate scheme optimized with reinforcement learning (HAS-RL) is proposed and we reduce the complexity of the HAS-RL by reducing the state space and action space, which will reduce the resource overhead as well. After that, we embed a global approximate controller in the NoC system, in which we deploy a policy network trained with the offline reinforcement learning algorithm to adjust the data approximate rates of each node at run time. Compared with the state-of-the-art method, the proposed scheme reduces the average network delay by 13.5% while their accuracies are similar. The proposed HAS-RL only causes an additional area overhead of 1.24% and power consumption of 0.77% compared with the traditional router design.

Data-intensive applications, such as machine learning and pattern recognition, result in heavy Network-on-Chip (NoC) communication loads and a tremendous increase in the communication latency. At the same time, the error-tolerant nature of these applications makes approximate communication an effective way to relieve the sharp increase of the network latency. This paper proposes an adaptive congestion-aware approximate communication mechanism (ACAC) that can alleviate the communication congestion of NoC systems in heavy communication loads. Our cycle-accurate simulations have shown that the proposed ACAC effectively reduces the network latency similar to ABDTR under a 22% to 52% lower data approximate ratio and significantly decreases the additional compression control traffic volume under real applications.

Hardware mapping plays a critical role in the performance of NoC-based accelerators running large-scale neural networks (NN). Confronted with enormous mapping exploration space, traditional algorithms may find sub-optimal solutions. We conduct preliminary experiments to investigate the impact of different hardware mappings on communication latencies. Then, this paper proposes an Autonomous Optimal Mapping Exploration (AOME) architecture based on two reinforcement learning algorithms. Combining soft and hard constraints, AOME transforms the mapping process into a sequential decision problem and targets to explore the optimal mapping of the NoC system. We evaluate the performance of AOME on ten NNs. The results show that compared with the direct X mapping, the direct Y mapping, GA-base mapping, and NN-aware mapping, AOME reduces the average communication latency of ten NNs by 27.30%, 33.33%, 4.27% and 12.46% using A2C, by 27.19%, 33.21%, 4.11% and 12.31% using PPO, and improves the average communication throughput by 43.24%, 63.60%, 5.17% and 14.83% using A2C, by 43.18%, 63.68%, 5.23% and 14.87% using PPO.