Knowledge distillation transfers knowledge from a complex network (teacher) to a lightweight network (student). Existing knowledge distillation methods typically employ a loss function comprising task and distillation losses, and they use a hyper-parameter to balance the two losses. However, in this paper, we observe an inconsistency between the gradient directions of these two losses, which introduces a trade-off between two gradients, hindering the student from learning the entire knowledge. To overcome this challenge, we propose a Universal Knowledge Distillation Framework Based on Decoupling Gradients (DGKD). DGKD breaks the trade-off by decoupling the gradients of the two losses and distributing these to separate branches of the student. Additionally, we introduce a Knowledge Interaction Module (KIM) and Inference-stage Simplifications to optimize our DGKD, enhancing its flexibility and simplicity. Extensive experiments validate the superiority of DGKD. For example, DGKD achieves a +2.80% accuracy improvement on CIFAR-100 for the VGG13-MN-V2 pair.

In modern operational environments, all work plans are susceptible to the influence of emergencies, such as equipment failures and resource shortages, underscoring the criticality of emergency management. Current solutions primarily rely on experienced human or employ machine scheduling models. Human-machine collaboration can leverage their respective strengths, thereby enhancing decision-making efficiency and safety. Based on this, a framework for human-machine collaborative emergency management is proposed in this paper. The emergency management process is divided into two stages: task selection and task scheduling. In the task selection phase, the Human-Machine Collaborative Decision-Making Algorithm for Dynamic Tasks (HMC-DMADT) is introduced to identify key nodes and generate task lists, and humans can correct and confirm lists they do not endorse. In the task scheduling phase, the Dynamic Task Scheduling Algorithm with Human Experience-Based Constraints (HEC-ETSA) is proposed, which integrates the Action-Mask mechanism with the Double Deep Q-Network (DDQN) algorithm to optimize action selection, ensuring decision safety and feasibility. Finally, a simulation platform is established to conduct numerous experiments within an unmanned warehousing scenario. The results demonstrate that the proposed human-machine collaborative emergency management framework effectively addresses emergencies in unmanned warehousing operations.

The Internet of Things (IoT) is a network composed of information-gathering devices, sensors, and computing devices assembled in an intelligent manner, and the most important element in this system is data. IoT data trading plays a vital role in the area of personalized business nowadays. Individual IoT devices generate large amounts of private data, which data owners can sell to enterprises as important digital assets to make money, while enterprises collect IoT data to improve the accuracy of their services. Cloud storage services have been widely used in IoT data trading. In IoT data trading, cloud storage services have been widely used for individuals to store IoT data and for enterprises to automatically facilitate data trading. However, there are still two crucial drawbacks to be solved. From the point of security, it is difficult for data buyers to check the validity of the search result without getting the decryption key of the data owner. From the point of fairness, there is a lack of a punishment mechanism to transfer money from the cheating party to the honest party. To tackle the two challenges, we propose a searchable re-encryption scheme to maintain traditional security without sacrificing service quality. Next, we design a fair trading protocol based on smart contracts to automatically detect any cheating behaviors. Formal security analysis proves that the scheme provides expected security. Experimental results show that the scheme achieve good performance.

Network pruning is one of the most dominant model compression methods today, which can be broadly divided into filter pruning and weight pruning. Unlike filter pruning that deletes the whole filters thus prone to cause unrecoverable accuracy loss, weight pruning removes the single weights at a fine-grained level, which effectively avoids this problem. However, weight pruning leads to unstructured sparsity and is thus incompatible with general platforms. To address such limitation, we propose Splittable Pattern-Specific Weight Pruning(SPWP) to achieve both compression and compatibility, consisting of Patterned Weight Searching(PWS) and Kernel Equivalent Splitting(KES). Specifically, we study the intrinsic features of convolution kernels and devise PWS to prune weights in regular shapes based on such features. During inference, KES equivalently splits the pruned sparse kernel into parallel branches according to the linear additivity of convolution, allowing the network to be accelerated on general platforms. Extensive experiments of different models on various datasets demonstrate the superior performance of our method. For example, SPWP can prune 60.1% total FLOPS of ResNet-56 on CIFAR-10 with even a 0.08% of top-1 accuracy increase.

Network pruning is effective in reducing memory usage and time complexity. However, current approaches face two common limitations. 1) Pruned filters cannot contribute to the final outputs, resulting in severe performance drops, especially at large pruning rates. 2) It requires time-consuming and computationally expensive fine-tuning to recover accuracy. To address these limitations, we propose a novel filter pruning method called Cluster Pruning (CP). Instead of directly deleting filters, CP reconstructs them based on their intra-similarity and removes them using the proposed channel addition operation. CP preserves all learned features and eliminates the need for fine-tuning. Specifically, each filter is distinguished by clustering and reconstructed as the centroid to which it belongs. Reconstructed filters are updated to prevent erroneous selections. After convergence, filters can be safely removed through the channel addition operation. Experiments on various datasets show that CP achieves the best trade-off between performance and complexity compared with other algorithms.

Weight pruning is a well-known technique used for network compression. In contrast to filter pruning, weight pruning produces higher compression ratios as it is more fine-grained. However, pruning individual weights results in broken kernels, which cannot be directly accelerated on general platforms, leading to hardware compatibility issues. To address this issue, we propose Shift Pruning (SP), a novel weight pruning method that is compatible with general platforms. SP converts spatial convolutions into regular 1 X 1 convolutions and shift operations, which are simply memory movements that do not require additional FLOPs or parameters. Specifically, we decompose the original K X K convolution into parallel branches of shift-convolution operations and devise the Differentiable Shift Operator (DSO), an approximation form of the actual shift operation, to automatically learn the crucial directions for adequate spatial interactions with the designed shift-related loss function. A regularization term is proposed to prevent redundant shifting, which is beneficial for low-resolution situations. To further improve inference efficacy, we develop a post-training transformation that can construct a more compact model. The introduced channel-wise slimming allows SP to prune in a hybrid-structural manner, catering for both hardware compatibility and a high compression ratio. Extensive experiments on the CIFAR-10 and ImageNet datasets demonstrate that our proposed method achieves superior performance in both accuracy and FLOPs reduction compared to other state-of-the-art techniques. For instance, on ImageNet, we can reduce 48.8% of total FLOPs on ResNet-34 with only 0.22% Top-1 accuracy drop.

Recently, the applications of deep neural network (DNN) have been very prominent in many fields due to its superior feature extraction performance. However, the high-dimension parameter model and large-scale mathematical calculation restrict the execution efficiency, especially for the Internet of Things (IoT) devices. Different from the previous cloud/edge-only pattern that brings significant pressure for uplink communication and device-only fashion that undertakes unaffordable calculation strength, we highlight the collaborative computation between the device and edge for DNN models, which can achieve a good balance between the communication load and execution accuracy. Specifically, a systematic on-demand co-inference framework is proposed to exploit the multi-branch structure, in which the pre-trained Alexnet is right-sized through early-exit and partitioned at an intermediate DNN layer. The integer quantization is enforced to further compress transmission bits. As a result, we establish a new Deep Reinforcement Learning (DRL) optimizer-Soft Actor Critic for discrete (SAC-d), which generates the exit point, partition point, and compressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an optimizer can well adapt to the complex environment like dynamic wireless channel and arbitrary CPU processing, and is capable of supporting the 5G URLLC. Real-world experiment on Raspberry Pi 4 and PC shows the effective performance of the proposed solution.

Abstract Precipitation forecast verification is essential to the quality of a forecast. The Gaussian mixture model (GMM) can be used to approximate the precipitation of several rain bands and provide a concise view of the data, which is especially useful for comparing forecast and observation data. The robustness of such comparison mainly depends on the consistency of and the correspondence between the extracted rain bands in the forecast and observation data. We propose a novel co‐estimation approach based on GMM in which forecast and observation data are analysed simultaneously. This approach naturally increases the consistency of and correspondence between the extracted rain bands by exploiting the similarity between both forecast and observation data. Moreover, a novel visualization and exploration framework is implemented to help the meteorologists gain insight from the forecast. The proposed approach was applied to the forecast and observation data provided by the China Meteorological Administration. The results are evaluated by meteorologists and novel insight has been gained.

The method of corrosion diagnosis for substation grounding grids based on large change sensitivity is proposed. By analyzing the node voltages, the large change sensitivity of the port voltages in response to the branch resistances is given and the nonlinear optimal model of corrosion diagnosis is established. The diagnosis equations based on the differential sensitivity were set up in traditional methods. Contrasting with them, the equations established in the paper is more appropriate for the real corrosion case. To insure the independency of diagnosis equations, the optimal selection method of test ports is put up. The effectiveness of algorithm is validated by the simulation of substation grounding grids.

A novel topology analysis method is presented to analyze the testability of grounding grids, and creditability is firstly proposed as an important index to evaluate the results. In this method, inaccessible nodes are eliminated by wye-delta transformation, followed by the calculation of testability and creditability index in the reverse direction. Simulation results are provided to prove this method.