Overlapping graph clustering is essential to understand the nature and behavior of real complex systems including human interactions, technical systems and transportation network. However, in addition of topological structure, many real-world networked systems contain spare factors, i.e., attributes of networks. Despite the considerable efforts that have been made in graph clustering, they only concentrate on the topological structure, which lack a profound understanding of cluster configuration on attributed graphs. To address this great challenge, in this article, we propose a new overlapping graph clustering algorithm by integrating the topological and attributive information into a cluster potential game (CPG). Firstly, a generalized definition of the utility function is provided, which measures the payoff of each node based on different node-to-cluster distance functions. It is worth mentioning that the model we proposed is able to associate with the classic ordinal potential game well. Then, we define the measures of both tightness and the homogeneity in each cluster, and introduce a novel two-way selection mechanism. The goal is to extend the flexibility of the cluster potential game, so that one can achieve a win-win situation between nodes and clusters. Finally, a distributed and heterogeneous multiagent system (DHMAS) is carefully designed based on a fast self-learning algorithm (SLA) for attributed overlapping graph clustering. Two series of experiments are implemented in multi-types datasets and the results verify the effectiveness and the scalability after the comparison with the most advanced approaches of literature.