Edge computing has emerged as a popular paradigm for handling heterogeneous tasks to improve computing capacity and promote quality of service (QoS). Multi-access edge computing can balance overall benefits while increasing the efficiency of edge servers. However, the design of computation offloading is challenging. This study provides a Graph Attention Network Reinforcement Learning (GATRL) framework to address the offloading challenge, which models devices and edge servers as digraphs and the state policy by graph properties to address the problem. The GATRL uses supervised learning and combines the GAT and DQN. We propose a method to identify the approximate solution to achieve fast convergence based on the data generated by Gaussian distribution relied on actual parameters. Compared to other methods, numerical results show that the proposed GATRL algorithm can achieve near-optimal performance and significantly reduce the off-load time. In addition, graph neural networks can adapt to more variable network environments, thus rapidly changing the model structure and giving offloading solutions.