Given the delay-critical nature of AI-driven industrial automation applications, industrial cyber-physical systems are evolving from centralized cloud automation to decentralized cloud-fog automation to reduce model inference delay. However, traditional centralized deep federated learning is not wellsuited for this evolution, primarily due to scalability and delay issues caused by centralized parameter synchronization. Thus, we introduce a decentralized deep federated learning (DDFL) architecture. While DDFL resolves scalability and delay concerns, decentralized parameter synchronization amplifies the delay imbalance impact caused by uneven computing and network loads. Additionally, traditional single-task load balancing approaches with fixed load balancing weights face challenges posed by diverse delay requirements across different model training tasks. To overcome these challenges, we formulate a hybrid multi-task Markov decision process with the objective of minimizing flexibly weighted computing and network load. We further propose a hybrid multi-task deep reinforcement learning (MTDRL) scheme based on the importance-weighted actor-learner architecture, which trains a hybrid-MTDRL decision model to select fog servers and paths with balanced loads suited to diverse delay requirements. Realistic trace-based simulation and testbed evaluation results demonstrate that hybrid-MTDRL outperforms benchmarks in load balancing and reducing training delay.