In the contemporary landscape of computationally intensive applications, Computing Power Network (CPN) offers a solution to enhance computational efficiency and cost-effectiveness by integrating and sharing computing resources. However, with the surge in task volume within multi-user environments, effectively scheduling these tasks to optimize system profit and delay presents a significant challenge. This paper introduces an optimization approach leveraging Deep Deterministic Policy Gradient (DDPG) to enhance CPN performance through task decomposition and computing path optimization. We initially construct a multi-layer CPN system model encompassing cloud computing, edge computing, and terminal device layers. Subsequently, we integrate a novel mechanism for convex optimization-based task decomposition, enabling intelligent subdivision of tasks into sub-tasks and dynamic allocation to suitable nodes within the network. Furthermore, we devise a Convex Optimization Task Decomposition-based Multi-Agent Deep Deterministic Policy Gradient (CO-MADDPG) algorithm, empowering multiple computing tasks as independent agents to learn and identify optimal offloading paths and computing nodes, thereby minimizing delay and maximizing system profit. A series of simulation experiments validate the effectiveness of the CO-MADDPG algorithm in handling concurrent tasks, demonstrating its capability to reduce task completion times, enhance system revenue, and maintain adaptability and stability across varying task demands.