Video analytics is considered the killer application of edge computing and has been successfully deployed across diverse domains. Yet, executing video analytics on mobile devices presents notable challenges owing to the considerable computational demands and frame rate requirements of DNN models. Current mobile inference frameworks often concentrate on enhancing model inference performance on the CPU or GPU, overlooking the potential of the Digital Signal Processor (DSP) – an emerging heterogeneous processor increasingly integrated into modern mobile processors. In this paper, we introduce COUPLE, an orchestration framework for video analytics on heterogeneous mobile processors, with the goal of optimizing real-time video analysis through the collaboration of CPU, GPU and DSP. To tackle the accuracy loss of DSP inference, we introduce the Anchor Frame Calibration mechanism, utilizing high-precision GPU inference results and frame similarities to mitigate accuracy loss on the DSP. Additionally, we design a lightweight progressive scheduler to distribute video frames to GPU and DSP, maximizing inference Average Precision (AP) under performance (i.e., frame rate) and power constraints. COUPLE has been implemented on the Qualcomm's Snapdragon 888 mobile SoC, extensive evaluation results demonstrate its efficacy in imnroving the inference performance and accuracy.

To identify dense and small-size pedestrians in surveillance systems, high-resolution cameras are widely deployed, where high-resolution images are captured and delivered to off-the-shelf pedestrian detection models. However, given the highly computation-intensive workload brought by the high resolution, the resource-constrained cameras fail to afford accurate inference in real time. To address that, we propose Hode, an offloaded video analytic framework that utilizes multiple edge nodes in proximity to expedite pedestrian detection with high-resolution inputs. Specifically, Hode can intelligently split high-resolution images into respective regions and then offload them to distributed edge nodes to perform pedestrian detection in parallel. A spatio-temporal flow filtering method is designed to enable context-aware region partitioning, as well as a DRL-based scheduling algorithm to allow accuracy-aware load balance among heterogeneous edge nodes. Extensive evaluation results using realistic prototypes show that Hode can achieve up to 2.01× speedup with very mild accuracy loss.

Deep learning has achieved tremendous success in various fields, but its significant computational demands make inference on mobile devices extremely challenging. To address this issue, we propose the COUPLE system, which enables heterogeneous processors to collaborate on mobile devices for accelerating video analytics. Additionally, we design the Co-Optimize strategy which utilizes the inference results of GPU to mitigate the accuracy loss caused by DSP. Experimental results demonstrate that COUPLE can improve the inference Average Precision by up to 5% compared to existing solutions.