Collaborative inference among Internet-of-Things (IoT) devices can reduce deep neural network (DNN) inference latency by exploiting the communication and computing resources of IoT devices. However, existing collaborative inference strategies often overlook the padding data integrity in information interaction among devices, leading to the loss of boundary data or redundant communication overhead, thus undermining the inference latency improvements. To address these issues, in this paper, we propose PIDC, a padding-aware IoT device collaboration framework for accelerating DNN inference, which jointly optimizes DNN partitioning and padding interaction among devices to minimize inference latency. First, the minimum amount of data exchanged for padding interaction is analyzed. Then, we formulate the latency minimization problem as a nonlinear integer programming problem, and transform it into a linear programming formulation by introducing auxiliary variables, enabling efficient solution with existing solvers. We implement a prototype using heterogeneous devices to validate the effectiveness of PIDC in real-world settings. Experimental results demonstrate that PIDC achieves significant inference latency reductions, with up to 43.0% latency reductions across different DNN models and datasets compared to the state-of-the-art methods.

Due to the limited energy and computing resources of Internet of Things (IoT) devices, the collaboration of IoT devices and edge servers is considered to handle the complex deep neural network (DNN) inference tasks. However, the heterogeneity of IoT devices and the various accuracy requirements of inference tasks make it difficult to deploy all the DNN models in edge servers. Moreover, a large-scale data transmission is engaged in collaborative inference, resulting in an increased demand on spectrum resource and energy consumption. To address these issues, in this paper, we first design an accuracy-aware multi-branch DNN inference model and quantify the relationship between branch selection and inference accuracy. Then, based on the multi-branch DNN model, we aim to minimize the energy consumption of devices by jointly optimizing the selection of DNN branches and partition layers, as well as the computing and communication resources allocation. The proposed problem is a mixed-integer nonlinear programming problem. We propose a hierarchical approach to decompose the problem, and then solve it with a proportional integral derivative based searching algorithm. Experimental results demonstrate our proposed scheme has better inference performance and can reduce the total energy consumption up to 65.3$\%$, compared to other collaboration schemes.