The rapid development of AI accelerates the implementation and delivery of AI applications in diverse fields. In cloud-edge collaboration, delivering a complete AI application relies on the robust coordination between AI-supporting microservices and AI services. However, most existing studies only coarsely considered monolithic AI service orchestration while neglecting microservice orchestration. Such coarse-grained orchestration severely impacts application performance. To enable diverse high-performance AI applications, fine-grained hybrid orchestration of AI services and microservices (HOAIM) is highly desirable, yet presents formidable challenges. Due to heterogeneous services, call dependencies, and service multiplexing, fine-grained hybrid orchestration modeling is highly non-trivial. Moreover, the tight coupling between deployment and routing results in a complex joint optimization problem. To address this, we first propose a heterogeneous service orchestration network that supports orchestration optimization and automated management. Then, based on queuing networks and multi-instance models, we conduct an accurate analysis of delay and load. Furthermore, to achieve efficient hybrid orchestration, we propose preference-driven resource allocation and instance computation algorithms, along with reinforcement learning with action masking and reward shaping. Finally, extensive trace-driven simulations demonstrate that our algorithms optimize average response delay by up to 41.83%, and achieve significant advantages in load balancing, response success rate, and resource efficiency.

The emerging microservice architecture offers opportunities for accommodating delay-sensitive applications in edge. However, such applications are computation-intensive and energy-consuming, imposing great difficulties to edge servers with limited computing resources, energy supply, and cooling capabilities. To reduce delay and energy consumption in edge, efficient microservice orchestration is necessary, but significantly challenging. Due to frequent communications among multiple microservices, service deployment and request routing are tightly-coupled, which motivates a complex joint optimization problem. When considering multi-instance modeling and fine-grained orchestration for massive microservices, the difficulty is extremely enlarged. Nevertheless, previous work failed to address the above difficulties. Also, they neglected to balance delay and energy, especially lacking dynamic energy-saving abilities. Therefore, this paper minimizes energy and delay by jointly optimizing microservice deployment and request routing via multi-instance modeling, fine-grained orchestration, and dynamic adaptation. Our queuing network model enables accurate end-to-end time analysis covering queuing, computing, and communicating delays. We then propose a delay-aware reinforcement learning algorithm, which derives the static service deployment and routing decisions. Moreover, we design an energy-aware dynamic frequency scaling algorithm, which saves energy with fluctuating request patterns. Experiment results demonstrate that our approaches significantly outperform baseline algorithms in both delay and energy consumption.