Emerging applications raise critical QoS requirements for the Internet. The improvements in flow classification technologies, software-defined networks (SDN), and programmable network devices make it possible to fast identify users’ requirements and control the routing for fine-grained traffic flows. Meanwhile, the problem of optimizing the forwarding paths for traffic flows with multiple QoS requirements in an online fashion is not addressed sufficiently. To address the problem, we propose DRL-OR-S, a highly scalable online routing algorithm using multi-agent deep reinforcement learning. DRL-OR-S adopts a comprehensive reward function, an efficient learning algorithm, and a novel deep neural network structure to learn appropriate routing strategies for different types of flow requirements. In order to enhance the generalization and scalability, we propose a novel graph-based actor-critic network architecture and a carefully designed input state for DRL-OR-S. To accelerate the training process and guarantee reliability, we further introduce an NN-simulator for efficient offline training and a safe learning mechanism to avoid unsafe routes during the online routing process. We implement DRL-OR-S under SDN architecture and conduct Mininet-based experiments using real network topologies and traffic traces. The results validate that DRL-OR-S can well satisfy the requirements of latency-sensitive, throughput-sensitive, latency-throughput-sensitive, and latency-loss-sensitive flows at the same time, while exhibiting great adaptiveness and reliability under the scenarios of link failure, traffic change, unseen large topology and partial deployment.

In recent years, context information has played an increasingly significant role in recommendation systems. With the rapid growth of portable sensor devices, lifelog data, such as mood, location, and daily activity, has been recorded and used for ubiquitous recommendation tasks. However, since the multi-modal lifelog data contains objective context information and subjective user labeling, it is challenging to record the lifelog thoroughly and perform personalized recommendations in real-time. In this work, we design a mobile application (App), LifeRec, to record multi-modal lifelog data and perform personalized recommendations by communicating with the remote server. The App helps users collect various lifelog information (e.g., location, diet, activity, and mood) and receive real-time recommendation with privacy protection and little effort. It is useful for lifelog data collection, user status monitoring, and various ubiquitous recommendation tasks. We examine LifeRec in a one-week field study with seven subjects. The users’ experience feedback and recording results show great usability and task completeness with our App.