Emerging intelligent applications based on accurate and timely stream analytics require real-time CNN inference of massive data continuously generated at the pervasive end devices. Due to the resource constraints, neither computing locally at end devices nor transmitting to remote servers is competent for computation-intensive CNN inference on large-volume images in real-time. Therefore, Collaborative Inference (CI), which conducts inference sequentially from the local device to the remote server with compressed intermediate inference data, is rapidly promoted. Due to the essential communication in collaboration, the CI efficiency is sensitive to network conditions, and will degrade under the unpredictable network fluctuations in practice, which may cause a severe delay in CI and degrade the responsiveness of stream analytics. For accurate and timely stream analytics in practical fluctuating networks, we present RTCoInfer, the real-time CI framework with run-time transmission adaption considering the network conditions. Specifically, we propose a novel Switchable CNN integrating CNNs with different compression rates on the partition layer for the run-time transmission adjustment, and construct a real-time controller determining the compression rate to maintain the real-time CI for stream analytics. Extensive experiments show that, compared with state-of-the-art methods, RTCoInfer achieves better efficiency and unprecedented resilience in real-time stream analytics.

The emerging edge-cloud collaborative Deep Learning (DL) paradigm aims at improving the performance of practical DL implementations in terms of cloud bandwidth consumption, response latency, and data privacy preservation. Focusing on bandwidth efficient edge-cloud collaborative training of DNN-based classifiers, we present CDC, a Classification Driven Compression framework that reduces bandwidth consumption while preserving classification accuracy of edge-cloud collaborative DL. Specifically, to reduce bandwidth consumption, for resource-limited edge servers, we develop a lightweight autoencoder with a classification guidance for compression with classification driven feature preservation, which allows edges to only upload the latent code of raw data for accurate global training on the Cloud. Additionally, we design an adjustable quantization scheme adaptively pursuing the tradeoff between bandwidth consumption and classification accuracy under different network conditions, where only fine-tuning is required for rapid compression ratio adjustment. Results of extensive experiments demonstrate that, compared with DNN training with raw data, CDC consumes 14.9× less bandwidth with an accuracy loss no more than 1.06%, and compared with DNN training with data compressed by AE without guidance, CDC introduces at least 100% lower accuracy loss.